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PREFACE 

This manual serves as the definitive hardware reference guide for system designs using the 80960MC 
processor. Hardware designers can use this manual as a guideline for developing microprocessor 
systems. Readers of this manual should be familiar with the operating principles of microprocessors 
and with the 80960MC data sheet. 

This manual presents the 80960MC system design from a hardware perspective. Other information 
on the software architecture, instruction set, and programming of the 80960MC processor can be 
found in the 80960MC CPU Programmer's Reference Manual. 

Together with the 80960MC Hardware Designer's Reference Manual, these publications provide a 
complete description of the 80960MC system for hardware and software designers. 

MANUAL ORGANIZATION 

The manual is divided into three parts. Part I describes a single processor hardware design using the 
80960MC processor with the local bus. Part II discusses a multiprocessor design using 80960MC 
processors and BXUs with the Advance Processor bus (AP-bus). Finally, Part III describes fault- 
tolerant system design. 

There are 15 chapters and an appendix. The first five chapters describe how to build a hardware 
system using a single 80960MC processor. 

* Chapter 1 briefly introduces the 80960MC component architecture. 

* Chapter 2 presents an overview of the 80960MC hardware system design, which includes 
a system configuration illustrating the various components that constitute an 80960MC 
system. 

* Chapter 3 describes the local bus and the interface to the 80960MC processor. This chapter 
includes detailed signal descriptions and discusses timing generation, arbitration, interrupt 
handling, and initialization. 

* Chapter 4 discusses techniques for designing memory subsystems. 

* Chapter 5 presents guidelines on how to interface I/O devices to the local bus 

The next four chapters describe how to build a multiprocessor hardware system using the Advanced 
Processor bus. 

* Chapter 6 provides an overview of a 80960MC multiprocessor system. 

* Chapter 7 focuses on the AP-bus. It describes the AP-bus transactions, AP-bus protocol, 
and AP-bus signal timing. 

* Chapter 8 shows how to interface to the AP-bus by using the BXU. This chapter includes a 
description of the BXU, diagnostic support functions, performance evaluation, and system 
considerations. 
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* Chapter 9 presents guidelines on the memory and I/O interface to a BXU. 

The final six chapters present guidelines for fault- tolerant designs. 

* Chapter 10 presents an overview fault- tolerant design using the 8G960MC processor and 
the BXU. 

* Chapter 1 1 describes confinement areas and detection mechanisms used in 80960 fault- 
tolerant designs. 

* Chapter 12 discusses the error reporting mechanism used in 80960 fault-tolerant designs. 

* Chapter 13 explains the recovery mechanism. 

* Chapter 14 shows how to initialize a fault-tolerant system and provides a system initializa- 
tion example. 

* Chapter 15 provides guidelin' on how to design fault- tolerant I/O subsystems. 

Appendix A contains the descriptions of the registers and commands of the BXU. 

Wherever appropriate, design examples are included in the chapters. These designs are based upon 
functional 80960MC boards and systems, and are simplified for ease of understanding. The 
simplified versions of these designs have not been tested except for the figures that show part 
numbers. 

NOTATION CONVENTIONS 

This manual uses the following style conventions . 

* Integer numbers are presented in decimal notation unless otherwise indicated by the 
subscript "H" for hexadecimal or "B" for binary. 

* An active low signal is represented by a line over the signal name. For example, READY is 
an active low signal. 

* Names of bits, address and data fields, and registers of the BXU are noted by a serif italics 
typeface (e.g., the Access bit, the Component-ID field, or the Prefetch-Control register). 

* Names of other items that do not refer to registers of the BXU, such as cache parameters, 
are noted by a sans serif italics typeface (e.g. the tag, address block, or way). 

* For the appendix, the name of the particular register or command described is listed on the 
page headings. 
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CHAPTER 1 
INTRODUCTION TO THE 80960MC MICROPROCESSOR 

The 80960MC is the military version of the 80960 family, designed especially for high reliability 
embedded applications. At an operating frequency of 20 MHz, this high performance processor can 
sustain an instruction execution rate of seven and one-half million instructions per second (MIPS), 
and burst rates of 20 MIPS*. The 80960MC processor enhances embedded system performance by 
integrating special features to eliminate the need for additional peripheral devices and the associated 
software overhead. For example, the 80960MC processor offers an on-chip floating-point process- 
ing unit, a memory management unit with virtual memory addressability, an improved interrupt 
handling capability, and support for multiple processors, multitasking, debugging, and tracing. 

This chapter describes the architectural attributes and enhancements of the 80960MC processor for 
embedded computing. 

ARCHITECTURAL ATTRIBUTES FOR EMBEDDED COMPUTING 

For over a decade, Intel has designed a large variety of 8 and 16 bit microcontrollers to fit the needs 
of embedded applications. Based on this experience, several architectural attributes shared by both 
microcontrollers and microprocessors, have been implemented that benefit embedded applications 
and enhance microprocessor performance. Because the 80960MC processor incorporates these at- 
tributes (listed below) in its architecture, embedded applications are easy to design, perform well, and 
get to market fast. 

• Simple load/store design 

• Large general-purpose register sets 

• Boolean and bit-field instructions 

• Small number of operations and addressing modes 

• Simplified instruction format 

• Minimum cycle operation 

Load/Store Design 

In the 80960 family architecture, operations are register-to-register, with only LOAD and STORE 
instructions accessing memory. This attribute simplifies the instruction set and shortens cycle time. 

The 80960MC processor uses LOAD and STORE instructions to access memory. It further 
minimizes accesses to memory by providing a 512-byte, direct-mapped instruction cache. When a 
memory access is required, the processor can perform a burst transaction that accesses up to four data 
words with one word transferred every clock cycle. 



: DEC VAX 1 1/780 equals 1 MIP. 
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Large General-Purpose Register Sets 

Because the instructions operate on operands within registers, the 80960 family uses many registers. 
The 80960MC processor features large, versatile register sets. For maximum flexibility, each 
processor provides thirty-two 32-bit registers and four 80-bit floating-point registers. 

There are two types of general-purpose registers: local and global. The processor automatically 
accesses the 16 local registers when a procedure call is performed. Multiple sets of local registers 
are stored on-chip to further increase the efficiency of this register set, as shown in Figure 1-1. The 
register cache holds up to four local register frames, which means that up to three procedure calls can 
be made without having to access the procedure stack resident in memory. 
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Figure 1-1 : Local Register Set 

The 20 global registers retain their contents across procedure boundaries. The global registers con- 
sist of sixteen 32-bit registers (G 15 through G ) and four 80-bit registers (FP 3 through FP ), as shown 
in Figure 1-2. While all registers can be used for floating-point operations, the 80-bit registers are 
used for accumulation of extended precision results. 

Small Number of Addressing Modes 

The 80960 family uses relatively few addressing modes to facilitate a fast, simple interpretation by 
the control engine. The 80960MC processor provides simple, fast addressing modes, as well as a few 
complex addressing modes to allow optimization for code density. 
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Simplified Instruction Format 

A simplified instruction format eases the hardwired decoding of instructions, which again speeds 
control paths. The 80960MC processor's instruction formats are simple and word aligned; all 
instructions are one word long except for one class that uses the subsequent word as a 32-bit 
displacement. To further enhance performance, the instructions do not cross word boundaries. This 
feature eliminates a pipeline stage (that would have to align instructions) and decreases instruction 
execution time. 
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Figure 1-2: Global Register Set 



Overlapped Execution 



To optimize performance, the 80960MC processor overlaps instruction execution by means of write 
buffering and register score boarding. Write buffering allows a write instruction to proceed as soon 
as it is placed in the buffer. It does not have to wait for the actual write operation to occur on the 
L-bus. 

Similarly, register scoreboarding is a design technique that allows the 80960MC to continue 
execution of instructions when it encounters a LOAD instruction. When the LOAD instruction 
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begins, the 80960MC sets a scoreboard bit on the target register. After the target register is loaded 
with data, the processor resets the bit. While the data is being retrieved, additional instructions that 
do not reference the target register can be executed. The 80960MC ensures that these additional 
instructions do not reference the target register by checking the scoreboard bit transparently (no 
software required). The scoreboard bit is a "safety feature" that protects the user in the event that the 
compiler attempts to use a register that is not yet loaded. Thus, the scoreboard feature reduces the 
effect of slow memory speed and provides a useful tool for optimizing procedures. 

Minimum Cycle Operation 

The 80960MC processor executes most of the core instructions in a single clock cycle. For these 
instructions, the 80960MC processor uses hardwired logic rather than microcode to execute the 
instruction. 

The 80960MC also supports a number of important multicycle instructions, such as 32-bit multiply 
and divide instructions. These auxiliary functions require more than one clock cycle because it is 
more efficient to use microcode than hardwired logic. On the other hand, the integration of these 
functions on-chip eliminates much software overhead and the negative effects on code density that 
would be otherwise required. Thus, the additional functionality of the 80960MC enhances overall 
system performance while keeping code size small. 

ADDITIONAL 80960MC ARCHITECTURAL ENHANCEMENTS 

The 80960MC incorporates useful features such as on-chip floating-point processing, multiproces- 
sing capabilities, hardware multitasking, interprocess communication, virtual memory, and debug- 
ging functions that support breakpoint instructions, tracing, procedures, branches,etc. 

Floating-Point Operation 

The on-chip floating-point unit of each processor improves the performance of floating-point 
calculations by eliminating bus overhead used to transfer operands to a coprocessor. The processor 
provides hardware support for both mandatory and recommended portions of IEEE standard 754 for 
floating-point arithmetic, exponential, logarithmic, and other transcendental functions. By integrat- 
ing the floating-point unit on-chip, the 80960MC processor reduces the overall chip count for a 
system, decreases power consumption, and increases overall performance and reliability. 

Debug Capabilities 

The processor provides extensive system debug capabilities, an important feature for embedded 
computing where the ability to instrument an application may be limited. The 80960MC processor 
allows breakpoint instructions that stop program execution on various events, such as procedure 
calls, or certain instructions. Another debug facility traces the activity of the processor while it is 
executing a program. Tracing is done by recording the addresses of instructions that cause trace 
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events to occur. For example, a trace event can occur on the execution of a specific instruction, 
branch, or procedure call. To ensure that the 80960MC is operating properly, the processor performs 
a self-test when it is reset. If the self-test is successful, the 80960MC begins operation, otherwise 
it enters the stopped state. 

Multitasking Programs 

The 80960MC processor supports hardware multitasking, interprocess communication, and multiple 
processor configurations. The 80960MC processor offers several hardware functions designed to 
support multitasking programs. One unique feature, called self-dispatching, allows a processor to 
switch itself automatically among scheduled tasks. When self-dispatching is used, the operating 
system only needs to place the task in a common interprocessor scheduling queue. 

Memory Management 

For multitasking applications that require software protection and a large address space, the 
80960MC processor provides a memory management unit. To ensure the highest level of 
performance possible and reduce chip count, the memory management unit and translation look- 
aside buffer are integrated on the chip. 

Interprocess Communication 

The 80960MC processor supports interprocess communication by using hardware recognized data 
structures, called communication ports and semaphores. These ports are used to exchange messages 
and parameters between processes. The 80960MC handles the message passing automatically once 
the ports are set up by the programmer. 

Multiple Processors 

The 80960MC processor offers several functions to coordinate the actions of multiple processors. 
First, the processors can pass messages to each other to initiate actions such as flushing a cache, 
stopping or starting another processor, or preempting a task. Second, a set of synchronization 
instructions help maintain the coherency of shared memory. These instructions permit several 
processors to modify memory at the same time while maintaining data integrity. 

The self-dispatching mechanism of the 80960MC previously described provides another means of 
support for multiple processor applications. This mechanism, in addition to being used in single- 
processor systems, provides the means to increase the performance of a system by simply adding 
processors. 

Finally, the 80960MC processors synchronize themselves automatically when they perform system 
operations. A protocol is defined for multiprocessing that provides a low level set of operations. For 
example, binding a task to a processor means locking the task control block. 
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STANDARD BUS INTERFACE 

The advanced features of the 80960MC processor are implemented using a performance-optimized 
bus interface. The processor uses a high bandwidth local bus (L-bus) that consists of two standard 
signal groups: a 32-bit multiplexed address/data path and control signals for data transactions. 
Because of the large amount of caching, the L-bus supports burst transactions that transfer up to four 
successive data words. Transactions on the L-bus can use 8, 16, and 32 bit data types and address up 
to 4 Giga(G) bytes of physical memory. Bus arbitration can be accomplished by simply using the hold 
request/hold acknowledge protocol. 

INTER-AGENT COMMUNICATION/COPROCESSOR CAPABILITIES 

The 80960MC processor offers a flexible way to manage interrupts. It accepts interrupts in one of 
three ways: by communicating with an external interrupt controller using the standard Interrupt/ 
Interrupt Acknowledge signals, by activating the on-chip interrupt controller, or by accepting an 
Inter- Agent Communication (IAC) message. This allows the 80960MC to act as a coprocessor on 
a shared bus with another CPU. 

SUMMARY 

The 80960MC processor optimizes embedded system performance by using a new 32-bit architec- 
ture. The 80960 family architecture includes a load/store design, large general purpose register sets, 
fast addressing modes, a simplified instruction format, and minimized instruction execution cycles. 

To further enhance system performance, the 80960MC processor provides floating-point operation, 
interrupt controller capabilities, debug functions, multitasking, memory management, interprocess 
communication, and multiple processor capability. By integrating these functions on-chip, the 
80960MC reduces the power requirements and overall chip count for a system. 

As a result of the 80960 architecture, the 80960MC processor provides unprecedented performance. 
For a speed selection of 16 MHz, it can sustain an instruction execution rate of over six million MIPS 
and burst rates of 16 MIPS, speeds comparable to that of super minicomputers. The high instruction 
execution rates are made possible through a innovative design that incorporates an on-chip 
instruction cache with burst- transfer capability. 
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CHAPTER 2 
80960MC SYSTEM ARCHITECTURE 

This chapter illustrates the flexibility and power of the 80960MC system architecture using the 
advanced 32-bit 80960MC processor. It examines system configurations from a general perspective 
to explain the design concepts. Subsequent chapters describe the details of the system design. 

OVERVIEW OF A SINGLE PROCESSOR SYSTEM ARCHITECTURE 

The central processing module, memory module, and I/O module form the natural boundaries for the 
hardware system architecture. The modules are connected together by the high bandwidth 32-bit 
multiplexed L-bus, which can transfer data at a maximum sustained rate of 42 Mega(M) bytes per 
second for an 80960MC processor operating at 16 MHz. 

Figure 2-1 shows a simplified block diagram of a possible system configuration. The heart of this 
system is the 80960MC processor, which fetches program instructions, executes code, manipulates 
stored information, and interacts with I/O devices. The high bandwidth L-bus connects the 
80960MC processor to memory and I/O devices. The 80960MC processor stores system data, 
instructions and programs in the local ROM and RAM memory. By accessing various peripheral 
devices through the I/O interface, the 80960MC processor directly supports communication with 
other ancillary subsystems. 

80960MC Processor and the L-Bus 

The 80960MC processor performs bus operations using multiplexed address and data signals and 
provides all the necessary control signals. For example, stand ard In tel control signals are provided, 
such as Address Latch Enable (ALE), Address/D ata St atus (ADS), Write/Read command (W/R), 
Data Transmit/Receive (DT/R), and Data enable (DEN). The 80960MC processor also generates 
byte enable signals that specify which bytes on the 32-bit data lines are valid for the transfer. 

The L-bus supports burst transactions, which access up to four data words at a maximum rate of one 
word per clock cycle. The 80960MC processor uses the two low-order address lines to indicate how 
many words are to be transferred. The 80960MC processor performs burst transactions to load the 
on-chip 512-byte instruction cache to minimize memory accesses for instruction fetches. Burst 
transactions can also be used for data accesses. 

To transfer control of the bus to an external bus master, the 80960MC processor provides two 
arbitration signals: hold request (HOLD) and hold acknowledge (HLDA). After receiving HOLD, 
the processor grants control of the bus to an external bus master by asserting HLDA. 

The 80960MC processor provides a flexible interrupt structure by using an on-chip interrupt 
controller, an external interrupt controller, or both. The type of interrupt structure is specified by an 
internal interrupt vector register. For a system with multiple processors, another method is available, 
called inter-agent communication (IAC) where a processor can interrupt another processor by 
sending an IAC message. 
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Complete details of the L-bus and bus operations are discussed in Chapter 3. 
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Figure 2-1 : Basic 80960MC System Configuration 

Memory Requirements 

The system memory can consist of a memory controller, Erasable Programmable Read Only 
Memory (EPROM), and static or dynamic Random Access Memory (RAM). The memory controller 
first conditions the L-bus signals for memory operation. It demultiplexes the address and data lines, 
generates the chip select signals from the address, detects the start of the cycle for burst mode 
operation, and latches the byte enable signals. 

The memory controller generates the control signals for EPROM, SRAM, and DRAM. In particular, 
it provides the control signals, multiplexed row/column address, and refresh control for dynamic 
RAMs. The controller can be designed to accommodate the burst transaction of the 80960MC 
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processor by using the static column mode or nibble mode featu res of the d ynamic RAM. In addition 
to supplying the operation signals, the controller generates the READY signal to indicate that data 
can be transferred to or from the 80960MC processor. 

Chapter 4 provides design guidelines for the memory controller. 

I/O Interface 

I/O components can be configured to allow the 80960MC processor to use most of its clock cycles 
for computational and system management activities. Time consuming tasks can be off-loaded to 
specialized slave-type components, such as the M8259A Programmable Interrupt Controller. Some 
tasks may require a master-type component, such as the 82586 Local Area Network Control. 

The interface circuit performs several functions. It demultiplexes the address and data lines, 
generates the chip select signals from the address, produces the I/O read or I/O writ e comman d from 
the processor's W/R signal, latches the byte enable signals, and generates the READY signal. 
Because these functions are the same as some of the functions of the memory controller, the same 
logic can be used for both interfaces. For master-type peripherals that operate on a 16-bit data bus, 
the interface circuit translates the 32-bit data bus to a 16-bit data bus. 

The 80960MC processor uses memory-mapped addresses to access I/O devices. This allows the CPU 
to use many of the same instructions to exchange information for both memory and peripheral 
devices. Thus, the powerful memory-type instructions can be used to perform 8-, 1 6-, and 32-bit data 
transfers. 

Chapter 5 describes design guidelines for the I/O interface by examining representative design 
examples. 

SUMMARY 

The basic hardware system configuration is modular and flexible. The processor, memory, and 
I/O form natural boundaries in the basic hardware system architecture. The high-bandwidth L-bus 
that supports burst transfers is used for the data path between the 80960MC processor and other 
system devices. 

This chapter presented an overview for basic hardware system design. The next three chapters 
discuss the details of the architecture's local bus, memory and I/O interface to the L-bus. 
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CHAPTER 3 
THE 80960MC PROCESSOR AND THE LOCAL BUS 

The 32-bit multiplexed local bus (L-bus) connects the 80960MC processor to memory and I/O and 
forms the backbone of any 80960MC processor based system. This high bandwidth bus provides 
burst-transfer capability allowing up to four successive 32-bit data word transfers at a maximum rate 
of one word every clock cycle. In addition to the L-bus signals, the 80960MC processor uses other 
signals to communicate to other bus masters. This chapter, which describes these signals and the 
associated operations, follows the outline shown below: 

L-bus states and their relationship to each other 

L-bus signal groups, which consist of address/data and control 

L-bus read, write, and burst transactions 

L-bus timing analyses and timing circuit generation 

Related L-bus operations such as arbitration, interrupt, and reset operations 

OVERVIEW OF THE 80960MC L-BUS 

The L-bus forms the data communication path between the various components in a basic 80960MC 
hardware system. The 80960MC processor utilizes the L-bus to fetch instructions, to manipulate 
information from both memory and I/O devices, and to respond to interrupts. To perform these 
functions at a high data rate, the 80960MC processor provides a burst mode, which transfers up to 
four data words at a maximum rate of one 32-bit word per clock cycle. The 80960MC L-bus has the 
following features: 

• 32-bit multiplexed address/data path 

• High data bandwidth relative to the speed selection of the 80960MC processor 

• Four byte enables and a four- word burst capability that allow transfers from 1 to 16 bytes in 
length 

• Support for TTL latches and buffers. 

BASIC L-BUS STATES 

The L-bus has five basic bus states: idle (T), address (T a ), data (T d ), recovery (T), and wait (T w ). 
During system operation, the 80960MC processor continuously enters and exits different bus states 
as shown in Figure 3-1. This state diagram assumes that only one bus master resides on the L-bus. 

The local bus occupies the idle (T) state when no address/data transfers are in progress. When a new 
request is received, the L-bus enters the T a state to transmit the address. 

Following a T a state, the L-bus enters a T d state to transmit o r receive data on the address/data lines 
provided that the data is ready (indicated by the assertion of READY at the input of the processor). 
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If the data is not ready, the L-bus enters a T w state and remains in this state until data is ready. T w 
states may be repeated as many times as necessary to allow sufficient time for the memory or I/O 
device to respond. 




READY 



NO REQUEST 



T, — IDLE STATE 
T.— ADDRESS STATE 
T d — DATA STATE 
T r — RECOVERY STATE 
T w — WAIT STATE 



READY — BEAQY ASSERTED 
NOT READY — READY NOT ASSERTED 
BURST — MULTIPLE WORD ACCESS IN PROGRESS 
NO BURST — MULTIPLE WORD ACCESS DONE, OR A 
ONE-WORD ACCESS 
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Figure 3-1 : Basic L-Bus States 



After a data word is transferred in a non-burst transaction, the L-bus exits the T. or T state and enters 

d w 



the recovery (T.) state. In the case of a burst transaction, the local bus will exit the T d or T w state and 
re-enter the T . state to transfer the next data word. Once all data words have been transferred in a 

d 

burst transaction (up to four), the L-bus enters the T state to allow devices on the L-bus to recover. 
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When the recovery state is complete the L-bus will enter the T. state if no new request is pending. 
If a request is pending, the L-bus will enter the T a state to transmit the new address. 

L-BUS SIGNAL GROUPS 

Signals on the L-bus, shown in Figure 3-2, consist of two basic groups: address/data, and control. 
A description of both of these signal groups is provided in this section. 
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Figure 3-2: L-Bus Signal Groups 

Address/Data 

The address/data signal group consists of 32 bi-directional active high lines. These signals are 
multiplexed to serve a dual purpose depending upon the bus state. 

LAD 31 -LAD LOCAL ADDRESS/DATA 31 through LOCAL ADDRESS/DATA 2 ) rep- 

resent the address signals on the L-bus during the T a state. LAD 2 is the least 
significant bit, and LAD 31 is the most significant address bit. LAD 31 through 
LAD 2 contain a physical word address. LAD 2 and LAD specify the number 
of data words to transfer within a burst transaction. The address/data signals 
float to a high impedance state when the L-bus is not acquired. 

During the T a state LAD 2 and LAD represent the SIZE signals indicate 
whether one, two, three, or four words are transferred during the current 
transaction. These signals are valid during the T a state of the L-bus. The 
encoding of the LADj and LAD signals to represent the size of a burst 
transaction is shown in Table 3-1. 

LOCAL ADDRESS/DATA 31 through LOCAL ADDRESS/DATA Q rep- 
resent the data signals on the L-bus during the T d and T w states. LAD is the 
least significant bit, and LAD 31 is the most significant data bit. The address/ 
data signals float to a high impedance state when the L-bus is not acquired. 
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Table 3-1 : SIZE Signal Decoding 



Word Selection 


LAD 1 


LAD 


1 Word 


Low 


Low 


2 Words 


Low 


High 


3 Words 


High 


Low 


4 Words 


High 


High 



Control 

The control signal group consists of 12 signals that control the transfer of data. These signals can 
be used to control data buffers, address latches, and other standard interface logic. 



ALE 



The ADDRESS LATCH ENABLE is an active low signal that can be used 
to latch the address from the 80960MC processor. ALE is asserted d uring 
the T a state and deasserted before the beginning of the T d state. The ALE 
signal floats to a high impedance state when the L-bus is not acquired. 



ADS 



ADDRESS STATUS is an active low signal that is driven by the 80960MC 
processor to indicate an address state. ADS is asserted during every T a state 
and deasserted during the following T d and T w states. For a burst transaction, 
ADS is asserted again every T d and T w state where READY was asserted in 
the prior cycle. The ADS signal is an open drain output. 



DT/R 



DATA TRANSMIT/RECEIVE indicates the direction of data flow to or 
frornthe local bus. For a read operation or an interrupt acknowledgement, 
DT/R is low during the T a , T d , and T w states to indicate that data flows into 
the 80960MC processor. For a write operation, DT/R is high during the T a , 
T d , and T w states to indicate that data fl ows from the 80960MC processor. 
DT/R never changes states when DEN is asserted. 



DEN 



DATA ENABLE is an active-low signal that can be used to enabl e data 
transceivers. DEN is asserted during all T. and T states. The DEN line is 
an open drain output of the 80960MC processor. 



W/R 



THE WRITE/READ signal instructs a memory or I/O device to write or 
read data on the L-bus. The 80960MC processor asserts W/R during a_T a 
state. The signal remains valid during subsequent T d and T w states. W/R is 
an open drain output of the 80960MC processor. 



BE 3 -BE 



THE BYTE ENABLE output signals of the 8G960MC processor specify 
which bytes (up to four) on the 32-bit data bus are transferred during the 
transaction. Table 3-2 shows the decoding scheme for these signals. 
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Table 3-2 : Byte Enable Signal Decoding 



Byte Enable Signal 


Address Line Selection 


BE, 


LAD 7 -LAD 


BE, 


LAD 16 -LAD 8 


BE, 


LAD 23 -LAD 16 


BB, 


LAD 31 -LAD 24 



The byte enable signals are valid from the 80960MC processor before data 
is transferred, as shown in Figure 3-3 (assumes no wait states). The byte 
enable signals that are valid for the first data word are specified during the 
T a state. For a four- word burst transaction, the byte enable signals that are 
valid for the second word are asserted during the first data state (T d0 ), for the 
third word during the second data state (T dl ), and for the fourth word during 
the third data state (T d2 ). The byte enable signals are undefined during the 
last data state (T d3 ) of the last word transferred. 
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Figure 3-3: Byte Enable Timing Diagram 

Although not shown in the diagram, the byte enable signals of each word are 
latched internally by t he 80960M C processor an d remain valid during every 
data or wait state until READY is applied. After READY is applied the byte 
enable signals change during the next T d state or become undefined for the 
last data transfer. 

The 80960MC processor asserts only adjacent byte enables. For example, 
the 80960MC processor does not perform a bus operation with only BE and 
BE 2 active. The Byte Enable lines are open drain outputs. 
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READY 



LOCK 



The READY signal indicates that the data on the L-bus can be sam pled 
(read) or removed (write) by the 80960MC processor. If READY is not 
asserted f ollowing the T a state or in between T d states, a T w state is generated. 
READY is an active-low input signal to the 80960MC processor. 

Bus LOCK prevents other bus masters from gaining control of the L-bus 
during a bus operation. It is activated by certain 80960MC processor op- 
eration and instructions. 



The 80960MC processor uses the bus LOCK signal when it performs a 
RMW memory operati on. W hen the processor performs a RM W-Read 
operation, it as serts th e LOCK signal during the T a state and holds LOCK 
asserted. If the LOCK signal was already asserted, the processor waits until 
this signal is deasserted before performing the RMW-Read operation. The 
processor deasserts the LOCK signal during the T a state when it performs 
a RMW- Write operation. 



The 80960MC processo r also as serts the LOCK signal during the interrupt 
acknowledge sequence. LOCK is an input and an open drain output signal 
from the 80960MC processor. 



CACHE 



The CACHE signal specifies whether the data is cacheable. If the 
80960MC processor asserts CACHE during the T a state, then the data is 
cacheable. The CACHE signal is undefined during the T d and T w states. The 
CACHE signal floats to a high impedance state when the L-bus is not 
acquired. 



Table 3-3 summarizes the L-bus signals. 

Additional pins are used by the 80960MC processor to control the execution of instructions and to 
interface to other bus masters. These pins include the arbitration, interrupt, error, and reset signals. 
Each of these signal groups are explained in separate sections. 



L-BUS TRANSACTIONS 

The 80960MC processor uses the L-bus signals to perform transactions in which data is transferred 
between the CPU and another component. During a transaction, the 80960MC processor can transfer 
up to four words of data to enhance system throughput. This is especially useful when loading cache 
menlory. 



3-6 



inter 



THE 80960MC MICROPROCESSOR AND THE LOCAL BUS 



Table 3-3: Summary Of L-Bus Signals 



Signal 
Group 


Signal 
Symbol 


Signal Function 


Active 
State 


Direction 


Type of 
Output 


Local 

Address/ 

Data 


Address 
(LAD 31 -LAD 2 ) 


32-bit word address 


T. 





3-state 


Size 
(LAD^LADo) 


Specifies number of 
words to transfer 


T. 





3-state 


Data 
(LAD 31 -LAD ) 


32-bit data 


1 d» 1 w 


I/O 


3-state 


Control 


SEE 


Enables address 
latch 


T. 





3-state 


ACS 


Identifies an address 
state 


T T< 1 >T < 1 > 
'a''d ''w 





Open drain 


DT/R 


Controls direction of 
data flow 


• •> • d» 1 w 





Open drain 


PER 


Enables data 
transceiver/latch 


T d ,T w 





Open drain 


W/R 


Write/read command 


1 »» 1 d» ' w 





Open drain 


BE3-EE0 


Specifies which data 
bytes to transfer 


T T (2 > T < 2 > 
'a»'d ''w 





Open drain 




Indicates data is 
ready to transfer 


T dl T w 


1 


— 


READY 


LOCK 


Locks bus 


Any 


I/O 


Open drain 


Cache 


Indicates cacheable 
transaction 


T. 





3-state 



NOTES: __ 

1 . Active after the first assertion of READY for a burst transaction. 

2. Active except for the last transfer. 

Clock Signal 

The 80960MC hardware system typically uses two clock signals, CLK2 and CLK, to synchronize 
the transitions between L-bus states. GLK2 is the clock input to the 80960MC and is double the 
specified processor frequency. CLK is an optional clock signal derived through external logic to 
provide a convenient reference of L-bus cycles and can be used to drive peripheral devices. CLK 
is one-half the frequency of CLK2, and is neither an input nor an output of the 80960MC processor. 
The convention utilized throughout this manual is CLK is low during the first half of the bus cycle 
and high during the second half. Figure 3-4 shows the relationship between the system CLK2 and 
CLK. 
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Figure 3-4: Clock Relationships 



Read Transaction 



Figure 3-5 shows a typical timing diagram for a read transaction (for exact timings, see the 80960MC 
processor data sheet). The following sequence of events explains the flow of the timing diagram. For 
simplicity, no wait states are shown. 

1. The 80960MC processor generates several signals during the T a state. 

• It transmits the address on the address/data lines. LADj and LAD specify a single word 
transaction. 



It asserts ALE. 



• It asserts ADS. 

• It asserts BE 3 -BE to specify which bytes are used when reading the data word. 

• It brings W/R low to denote a read operation. 

• It brings DT/R low. 

2. During the T state, several actions occur. 



• The 8096 0MC processor asserts DEN. DEN can be used to enable data transceivers. 
READY i s asserted by external timing logic and data is transmitted from the storage 
devi ces. If RE ADY is not asserted, the L-bus will enter a T w state. The T w state is repeated, 
until READY is asserted. 

• The 80960MC processor reads the data on the address/data lines. 

The T r state follows the data state. This allows the system components adequate time (one 
processor clock cycle) to remove their outputs from the bus before the 80960MC processor 
generates the next address on the address/data lines. 
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Figure 3-5: 80960MC Processor Read Transaction 



Write Transaction 



Figure 3-6 shows a typical timing diagram for a write transaction with one wait state. The following 
sequence of events explains the flow of the timing diagram. 

1 . Similar to the read transaction, the 80960MC processor generates several signals during the Ta 
state. 

• It transmits the address on the address/data lines. LAD t and LAD specify a single word 
transaction. 



• It asserts ALE. 



• It asserts ADS. 

• It asserts BE 3 -BE to specify which bytes are used when writing the data word. 
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• It brings W/R high to denote a write operation. 

• It brings DT/R high. 

2. During the T d state, several actions occur. 

• The 80960MC places the data on the address/data lines. 



The 80960MC processor asserts DEN. DEN can be used to enable data transceivers. 



READY is not asserted by external timing logic. Consequently, data is held on the LAD 
lines. 



3. During the_T w st ate RE ADY is asserted and the data is written t o the stor age device. Note that 
W/R, DT/R, and DEN remain constant until the bus state after READY is asserted. 

4. The T state follows the wait state. 
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Figure 3-6: 80960MC Processor write Transaction 
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Burst Transactions 

The 80960MC processor supports burst transactions that read or write up to four words (16 
contiguous bytes) at a maximum rate of one word every L-bus cycle. The byte enable signals are valid 
for each word to allow partial- word write operations to contiguous bytes within a word. The CACHE 
output signal during a T a state applies to all words of a burst transaction. 

A burst read or write transaction is similar to a single word read or write operation. It differs primarily 
in the number of data words transferred: the basic transaction always transfers one data word, the 
burst transaction transfers up to four data words. For a burst transaction, the byte enable signals are 
applied during the T a state, and subsequently during every T d or T w state before the data word is 
transferred. Figure 3-7 shows the timing for a three-word burst read transaction without wait states. 
Figure 3-8 shows the timing for a two- word burst write transaction with a wait state occurring during 
the tr ansfer of the first word. Note that the byte enable signals remain constant until the data state 
after READY is asserted. 
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Figure 3-7: 80960MC Processor Burst Read Transaction 
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Figure 3-8: 80960MC Processor Burst Write Transaction 

TIMING GENERATION 

In an 80960MC processor-based system, timing signals must be generated for the clock and reset 
inputs. To generate these signals, logic should be utilized to minimize skew and maintain the rise 
and fall times as short as possible. This section describes a typical circuit that synthesizes the clock 
signal. RESET timing generation is discussed in the "RESET AND INITIALIZATION" section of 
this chapter. 

Clock Generation 

Figure 3-9 shows an example of a clock generator that produces two clock pulses, one double the 
frequency of the other with the skew between the pulses in the range of 1 to 3 ns. This particular circuit 
produces a 32-MHz clock at a 50% duty cycle, The circuit design consists of four devices: an 
oscillator, a pulse shaping network, a synchronous up/down counter, and a NAND gate driver. The 
output of the 64-MHz hybrid clock oscillator connects to the pulse shaping network (two NAND 
gates in series), which in turn feeds into the clock input of the up/down counter. This counter 
produces a 32-MHz CLK2 output signal and a 16-MHz CLK output signal. Because the outputs of 
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the counter are synchronous, the skew between CLK2 and CLK is typically less than 2 ns. To provide 
adequate signal margin and maintain fast rise and fall times, the two clock signals are conditioned 
by the NAND gate driver. The timing waveforms of the clock circuit are shown in Figure 3-10. 

If the opposite phase CLK is preferred, the U/D pin can be connected to V cc . 
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Figure 3-9: Clock Generation Circuit 
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Figure 3-10: Clock Timing Waveforms 
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The hybrid clock oscillator typically requires 50 ms to stabilize after power is applied. The 80960MC 
processor cannot begin to execute instructions until after the plock and V GC have reached their DC 
and AC specifications. The RESET signal can be used to control the start of the CPU execution when 
power is applied. This is discussed in the "RESET AND INITIALIZATION" section of this chapter. 

ARBITRATION 

When multiple bus masters exist, an arbitration protocol is used to exchange control of the bus. The 
protocol assumes that there are two bus masters: one that controls the bus by default, and the other 
that requests control of the bus when it performs an operation, such as a DMA controller. More than 
two bus masters may exist on the L-bus, but this requires external arbitration logic. However, no 
more than two 80960MC processors may reside on an L-bus. 

This section examines bus arbitration, bus states, and timing diagrams for different combinations of 
two bus masters, as shown in Table 3-4. 

Single 80960MC Processor On The L-Bus 

For the first case, the 80960MC processor controls the L-bus, and a master I/O peripheral, such as 
a DMA controller, requests control of the bus. The 80960MC processor and the I/O peripheral 
exchange control of the bus with two signals: the hold request (HOLD) and hold acknowledge 
(HLDA) signals. 

HOLD is an input signal of the 80960MC processor, which indicates that the master I/O peripheral 
is requesting control of the L-bus. When HOLD is asserted, the 80960MC processor surrenders 
control of the bus after it completes the current bus transaction. The 80960MC processor 
acknowledges transfer of control of the L-bus to the requesting bus master by asserting HLDA. 

State Diagram 

Figure 3-11 shows the state diagram for an L-bus with two bus masters: an 80960MC processor, and 
an I/O peripheral device. This state diagram includes a hold state (T h ) in addition to the five basic 
states described in the "BASIC L-BUS STATES" section of this chapter. The 80960MC processor 
enters the T state when it surrenders control of the bus. It can enter the T state from the T.,T ,T ., 

n h v r 7 a' 

or T w state. When the 80960MC processor regains control of the L-bus, it enters the T a state if a new 
request is pending or a T state if no new request is pending. 
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Table 3-4: Combination of Bus Masters 





Bus Master Combination 




Bus Master that Controls the Bus 
by Default 


Bus Master that Requests 
Control of the Bus 


Case 1 


80960MC processor 


I/O device 


Case 2 


80960MC processor 


80960MC processor 


Case 3 


I/O device 


80960MC processor 
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REQUEST PENDING 
NO HOLD 




T. - IDLE STATE 

T a - ADDRESS STATE 

T d - DATA STATE 

T r - RECOVERY STATE 

T w - WAIT STATE 

T„ - HOLD STATE 



READY - READY ASSERTED 

NOT READY- READY NOT ASSERTED 

BURST - MULTIPLE WORD ACCESS IN PROCESS 

NO BURST - MULTIPLE WORD ACCESS DONE, OR A 

ONE-WORD ACCESS 
• = LOGICAL "AND" FUNCTION 
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Figure 3-11 : L-Bus States with Arbitration 



Arbitration Timing 



Figure 3-12 shows the arbitration timing diagram. The initial "T" state represents a T d , T w , T r , or T. 
state of a previous transaction as specified in the L-bus state diagram shown in Figure 3-11. The 
80960MC processor receives a request to relinquish control of the bus when HOLD is asserted. After 
the 80960MC processor completes the current transaction, it responds to this request by floating the 
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three-state output signals and deasserting the open drain output signals. The HLDA output signal, 
however, remains active and is asserted as the 80960MC processor enters a T h state. During the T h 
state, the CPU ignores all input signals except HOLD and RESET. When the HOLD input signal 
is deasserted, the 80960MC processor exits the T h state, deasserts HLDA, and enters a T a state if a 
request is pending or a T. state if no request is pending. 
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Figure 3-12: Arbitration Timing Diagram For A Bus Master 

Two 80960MC Processors On The L-Bus 

For the next case, two 80960MC processors reside on the L-bus. During initialization, Local 
Processor Number Zero (LPNO) is designated as the Primary Bus Master (PBM), and Local 
Processor Number One (LPN1) is designated as the Secondary Bus Master (SBM). The exchange 
protocol that is used guarantees that neither device is kept off the bus indefinitely. 

The 80960MC processors use two pins for bus arbitration: the HOLD input pin, and the HLDA 
output pin. However, these input and output pins are interpreted differently for the Secondary Bus 
Master. When the SBM is initialized, the pin normally used for the HOLD input signal is interpreted 
as the Hold Acknowledge Request (HLDAR) input signal. The assertion of HLD AR indicates that 
the PBM relinquished control of the L-bus. Similarly, the HLDA output signal of the SBM is 
interpreted as the hold request (HOLDR) output signal. The SBM asserts HOLDR to request 
acquisition of the L-bus. Thus, bus arbitration between two 80960MC processors can be accom- 
plished by connecting HOLD of the PBM to HOLDR of the SBM, and HLDA of the PBM to the 
HLDAR of the SBM, as shown in Figure 3-13. 

When using the connection shown in Figure 3-13, a delay must be inserted between the input and 
output signals because the minimum output float delay, (shown as T9 in the 80960MC Data Sheet), 
is less than the minimum hold time of the input signals, (shown as Tl 1 in the 80960MC Data Sheet). 
The delay time necessary to meet the specified input setup and hold times can be calculated by using 
the following equation: 

Tll(min) - T9(min) < Delay < 2*Tl(min) - T12(min) - T6(max) 



3-17 



iiteT 



THE 80960MC MICROPROCESSOR AND THE LOCAL BUS 



The "T" numbers used in the above equation refer to timing parameters listed in the 80960MC Data 
Sheet. Consult the data sheet for actual timing values. 
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Figure 3-13: Arbitration Connection Between Two 80960MC Processors 

Bus States For Two 80960MC Processors 

Th e state d iagram for the SBM is shown in Figure 3-14. Because there are two 80960MC processors, 
the LOCK signal is included in the state diagram. The SBM requests control of the L-bus by asserting 
HOLDR and subsequently enters the hold request (T hr ) state provided that the bus is not locked 
(lo cked me ans that the PBM is Currently executing a Read-Modify- Write operation and has asserted 
the LOCK signal). The SBM remains in the T hr state until it acquires control of the L-bus by receiving 
HLDAR. The SBM returns to the T state and deasserts HOLDR if the PBM asserts LOCK to execute 
a Read-Modify-Write operation. 

The SBM gains control of the bus when HLDAR is asserted provided that the bus is not locked. After 
gaining control of the L-bus, the SBM performs the requested transaction and, if necessary, enters 
a T w state. At the end of a transaction, the SBM goes to the T state and deasserts HOLDR for at least 
one processor clock cycle to allow another peripheral bus master to gain access if needed. If another 
request is pending, the SBM enters the T hr state and asserts HOLDR provided the bus is not locked. 
If no request is pending the SBM returns to the T state. The PBM never forces the SBM off the bus. 

Arbitration Timing for Two 80960MC Processors on the L-Bus 

Figure 3-15 shows the timing diagram for acquiring and relinquishing the L-bus by a SBM. The SBM 
enters into the Hold Request (T hr ) state and asserts the HOLDR signal. It remains in the T hr state until 
HLDAR is asserted, which indicates that the SBM has gained control of the L-bus. At the end of the 
transaction, the SBM enters the T state and deasserts HOLDR. Except for HOLDR, the output 
signals of the SBM go into a high impedance state or are deasserted for the case of open-drain outputs. 
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READY-BURST 




T f - IDLE STATE 

T fl - ADDRESS STATE 

T d - DATA STATE LOCKED 

T r - RECOVERY STATE 

T hr - HOLD REQUEST STATE HLDAR 

BURST 
NO BURST 



NO 
REQUEST 
+ LOCKED 

READY - READY ASSERTED 

NOT READY - READY NOT ASSERTED 



LOCK ASSERTED BY ANOTHER BUS MASTER AND 

RMW OPERATION PENDING FOR SECONDARY BUS MASTER 

HOLD ACKNOWLEDGE REQUESTED (REQUEST FOR BUS 

GRANTED) 

MULTIPLE WORD ACCESS IN PROGRESS 
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Figure 3-14: L-Bus States For Secondary Bus Master 
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Figure 3-15: Arbitration Timing Diagram For SBM 

Bus Exchange Example Between Two 80960MC Processors 

Figure 3-16 shows an example of bus arbitration between a PBM and a SBM using the arbitration 
signals. Each bus master performs a one- word read and a two-word write transaction to demonstrate 
the fastest possible bus exchanges. 
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Figure 3-16: Examble of a Bus Exchange Transaction 
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While the PBM is performing a read transaction, the SBM requests control of the L-bus by asserting 
HOLDR and entering the T hr state. It remains in this state until the PBM grants the request by 
asserting HLDA after the read transaction is completed. After granting the request, the PBM enters 
the Th state and remains in this state until its HOLD signal is deasserted. When the SBM completes 
the read transaction, it deasserts HOLDR and gives control back to the PBM. 

The PBM now performs a two-word write transaction after deasserting the HLDA. The SBM 
requests control of the bus again by asserting the HOLDR signal and enters the T hr state. When the 
PBM completes the two- word write transaction, it grants the request by asserting HLDA and enters 
the T h state. The SBM receives the signal on the HLDAR input and performs a two-word write 
transaction. When the SBM completes the transaction, the control of the L-bus is transferred to the 
PBM, and both the PBM and the SBM enter the T state. 

A Peripheral Device As The Default Bus Master 

Another case exists where a peripheral device controls the L-bus, and the 80960MC processor 
requests control of the bus to perform operations. This alternative is not advisable because it hinders 
system performance. The exchange protocol is identical to the one described in the previous section. 
The 80960MC processor is a SBM and uses two pins for bus arbitration: the HOLDR input pin and 
the HLDAR output pin. The state diagram is similar to the one shown in Figure 3-14. The lock 
conditions are not used for this case, however. 

The peripheral device grants control of the L-bus by asserting HLDAR when the SBM requests use 
of the L-bus. The peripheral device can obtain control of the L-bus again by deasserting HLDAR. 
If this occurs, the 80960MC processor surrenders control of the bus after it completes the current 
transaction, as shown in Figure 3-17. At that time, the 80960MC processor deasserts the HOLDR 
signal and places the other output signals into a high impedance state or a deasserted open drain level. 
The 80960MC processor may request access to the L-bus by asserting HOLDR again. 
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Figure 3-17: Forced Relinquishment Timing Diagram For A SBM 



3-21 



iny 



THE 80960MC MICROPROCESSOR AND THE LOCAL BUS 



INTER-AGENT COMMUNICATION (IAC) 

The IAC mechanism gives 80960MC processors the capability to send and receive messages to one 
another and to other bus agents. The IAC mechanism is essentially a NON-MASKABLE interrupt 
with pre-defined service routines. These routines are implemented in the 80960MC processor and 
are used to perform control functions such as purging the instruction cache, setting breakpoint 
registers, or stopping and starting the processor. By using IAC messages, external devices can 
remotely control the 80960MC. This allows easy integration of the 80960MC into system 
environments. 

IAC messages can also be used to generate interrupts that behave exactly the same as hardwired 
interrupts. Since the interrupt vector is encoded in the IAC message, any of the 248 possible interrupt 
service routines can be invoked. For further information on IAC message definitions see the 
80960MC Programmer's Reference Manual. 

Overview Of IAC Operations 

Figure 3-18 shows a typical example of an IAC operation. In this case, an external processor gains 
control of the 80960MC by using an IAC operation. The external processo r perf orms two functions: 
it writes the message in a buffer, c alled the message buffer; and it asserts the IAC pin of the 80960MC 
processor. Upon receipt of the IAC signal, the 80960MC processor stops executing its current 
process and performs a four-word burst read of the message buffer. After completing the read 
operation, the 80960MC processor automatically performs a one- word write operation to a pre- 
defined address to acknowledge the receipt of the message. The 80960MC processor then proceeds 
to perform the required action. When an IAC sender and recipient are the same physical processor 
(i.e. an 80960MC sending an IAC to itself), no L-bus activity is required. This allows software to 
move data or address ' s over the L-bus during the same bus cycle that it is executing the IAC, therefore 
increasing the effective throughput of the system. 

Hardware Requirements For IAC Messages 

To use the external IAC feature of the 80960MC, the following items are needed: a four- word 
Message Buffe r RAM mapped to a reserved address to store the messa ge, IA C notification logic to 
assert the IAC pin of the 80960MC, and decoding logic to deassert the IAC pin on command from 
the 80960MC. 

Message Buffers 

Each 80960MC processor that receives an IAC message must have four 32-bit words of message 
buffer. This buffer can use special hardware or a reserved area in RAM. For proper operation of the 
buffer, two requirements must be met: the receiving 80960MC must be able to read this buffer at 
FF000010 H if the receiving 80960MC's Local Processor Number (LPN) is equal to zero (see the 
"RESET AND INITIALIZATION" section of this chapter for details of the LPN), or at FF000030 H 
if the LPN is equal to one; and the sending processor must be able to write to this buffer. 
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Figure 3-18: Example Flow Chart For An I AC Operation 



I AC Pin Logic 



When the IAC message buffer receives a message, IAC notification logic asserts the IAC pin and 
keeps it asserted. After the 8G960MC processor reads the IAC message, it performs a one-word write 
to address FF000000H if its LPN is zero, or FF000020H if its LPN is one. This reserved address 
serves two functions: it causes external logic to deassert the IAC pin, and it maps to a register that 
contains the current processor priority. The 80960MC expects to write its priority into a 5-bit field 
of address FF000000 if its LPN is zero, or FF000020 if its LPN is one. To set the priority, the 
processor performs a one-word write operation in the form shown in Figure 3-19. The priority is 
contained in bit20-bitl6, and bit3 is asserted to indicate that the priority is changed. It is necessary 
to use bit3 as a qualifier to distinguish priority write operations from IAC message acknowledgments, 
which use the same reserved address. 
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Figure 3-19: Data Words 
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If the low order thre e bits of the data word have a value of 1 00B (see Figure 3-19), the external logic 
should deassert the IAC pin on completion of the write operation. 

EXTERNAL PRIORITY REGISTER 

The 80960MC keeps track of the current priority (a value between and 3 1) at which it is executing. 
This priority is used to decide whether or not to service interrupts ~ higher priority interrupts are 
serviced, others are posted for later servicing. In some system designs it may be desirable to have 
this priority visible outside of the processor. To allow this, the 80960MC provides IAC message 
support for an external priority register mapped to address FF000000 if its LPN is zero, or FF000020 
if its LPN is one. Whenever the priority of the 80960MC changes, the contents of this register are 
automatically updated (if enabled). 

This feature may be enabled in two steps. If the Write External Priority bit is set in the PRCB (see 
the 80960MC Programmer' s Reference Manual), then the external priority register is updated as a 
result of a process switch, an interrupt not caused by an IAC message, or the execution of a MODPC 
instruction (modify process controls). If external IAC messages are enabled, then the external 
priority register is also updated whenever a result of an IAC is to change processor priority. 

External Priority And IAC Messages 

Th e exte rnal priority register can be used to filter IAC messages. Since the processor always services 
the IAC pin (i.e., it is non-maskable), a low priority IAC message can interrupt a high priority task. 
To prevent this, a system can associate a priority with each IAC message. This priority can then be 
compared to the priority stored in the external priority register and used to decide whether or not to 
accept the IAC message. One way to associate a priority with an IAC message is to encode the 
message priority into the IAC message destination address as shown in Figure 3-20. The range of 
reserved addresses shown in Figure 3-20 have been set aside for this purpose. 
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Figure 3-20: Physical Address Interpretation For IAC Messages 

INTERRUPTS 

The 80960MC processor responds to external events occurring at arbitrary times by means of an 
interrupt signal. Various sources, which include hardware components and special software 
instructions, generate an interrupt signal that can suspend execution of the 80960MC processor's 
current instruction stream. Hardware-generated interrupts are discussed in this section. For 
complete information on software-generated interrupts, see the 80960MC Programmers Reference 
Manual. 
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The 80960MC architecture provides a flexible interrupt structure. The processor can be interrupted 
using any of the three methods shown below: 

• Receipt of a signal on any of the four direct interrupt input signal lines (INT , INT p INT 2 , and 
INT 3 ) 

• Receipt of a signal on the interrupt request (INTR) line to obtain an external interrupt vector 

• Receipt of an IAC message from a processor program or external source. 

The choice of the method is determined by the setting in the on-chip Interrupt Control Register. 
Interrupt signals can occur during any bus state regardless of which method is implemented. 

This section provides details on the multiplexed interrupt pins, the three interrupt methods, the 
Interrupt Control register, synchronization, and interrupt latency. 

Interrupt Signals 

The interru pt signals are multiplexed on four pins of the 80960MC processor: INT^AC, INT 1? INT 2 / 
INTR, and INT 3 /INTA. The on-chip Interrupt Control register determines how these pins are used 
(see "Interrupt Control Register" section of this chapter). 



INT/1AC 



This pin multiplexes the Interrupt and Inter-agent Communication 

(IAC) request in put si gna ls. Th e 809 60MC processor interprets this input 
signal as either INT or IAC. The IN T n sig nal indicates a reques t for 
interrupt service when it is asserted. The IAC signal denotes that an IAC 
message is waiting when it is asserted. 



INT, 



The Interrupt input signal indicates a request for interrupt service when it 
is asserted. 



INT 2 /INTR 



This pin multiplexes the Interrupt and Interrupt Request input signals. 
The 80960MC processor interprets this input signal as either INT 2 or INTR. 
The INT 2 signal indicates a request for interrupt service when it is asserted. 
The INTR signal indicates an interrupt request from an external interrupt 
controller. The 80960MC processor responds with an interrupt-acknowl- 
edge sequence. To ensure an interrupt, the INTR signal must remain 
asserted until the first cycle of the interrupt-acknowledge transaction. 



INT3/INTA 



This pin multiplexes the Interrupt input signal and Interrupt Acknowl- 
edge output sig nal. T he 80960MC processor uses this pin as the INT 3 input 
signal or as the INTA output signal. The Interru pt Contr ol Re gister s etting 
selects either the combination of INTR/INTA or INT2/INT 3 . The INT 3 
input signal indicates a request for interrupt service when it is asserted. 
INTA acknowledges the interrupt request from an external interrupt con- 
troller. The INTA signal is latched by the 80960MC processor and remains 
valid during the T d state and, if required, T w states. This signal is an open 
drain output. 
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Interrupt Control Register 

The 80960MC processor uses a 32-bit, on-chip Interrupt Control Register to define the function of 
the rnaltiplexed interrupt pins. T his 32 -bit Interrupt Cont rol R egister allocates eight bits for each of 
the four direct interrupt signals (INT , INT p INT 2 , and INT 3 ). The eight bits contain the vector 
number for each interrupt signal, a s sho wn in Figure 3-2 1 . Th e vector number is automatically read 
when one of the interru pt sig nals (INT n , INT,, INT 2 , arid INT 3 ) is activated. For example, when an 
interrupt is signaled on lNT , the 80960MC processor uses bit 7 -bit of the Interrupt Control register 
as the vector number. 
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Figure 3-21 : Interrupt Control Register 

The 8 Q96Q MC processor uses the data field cor respo nding to INT to determine identification of the 
INTq/IAC input pin; a value of 00 H signifies the IAC function. If the data field corresponding to INT 2 
has a va lue of 00 H , t he 8Q96QM C proc essor interprets the INT 2 /INTR pin as the INTR input signal, 
and the INT 3 /INTA pin as the INTA output signal. In other words, this setting specifies that the 
80960MC processor should use these two pins for communication with an external interrupt 
controller. When used with an external interrupt contro ller, the data field corresponding to INT^ 
should be set to FF H . If the functions of INTR and INTA are selected, the direct interrupt pins INT 
and INTj can still be used. 

The on-chip Interrupt Control register may be read and written by the Synchronous Load (synld) and 
Synchronous Move (synmov) instructions at the address FF000004 H (see the 80960MC Programmer' s 
Reference Manual). The value of the data fields in the Interrupt Control re gister i s FF000000 H after 
initia lization. This setting specifies that the four interrupt pins function as INTA, INTR, INT p and 

iac.-- ■■■•;; /; ' v"^ -■>■■ ■■ ; ' ^ ■■ ^ ■■ 

Using The Four Direct Interrupt Pins 

The 80960 MC pr ocessor can be interrupted by asserting any of the four interrupt input sign als (IN T n , 
INTj, INT 2 , INT 3 ). ilf the signals are simultaneously asserted, the 80960MC assumes that INT has 
the highest priority, followed by INT 1? INT 2 , and INT 3 . Software should follow this convention when 
programming the J^ter^upt Control register. When the interrupt input signals are asserted, the 
80960MC processor utilizes a vector number specified by the Interrupt Control register as an index 
to an entry in the interrupt t^ble located in memory. For complete software information on this topic, 
see the, 80960MC 'Programmer s Reference Manual. 



a-26 



irteT 



THE 80960MC MICROPROCESSOR AND THE LOCAL BUS 



Using An External Interrupt Controller 

The 80960MC processor can communicate with an externa l interrupt controller by performing an 
interrupt acknowledge sequence using the INTR and INT A signals. Figure 3-22 shows an example 
of the timing of an interrupt acknowledge sequence using the M8259A Programmable Interrupt Con- 
troller. 

INTR is asserted by the M8259A and remains asserted until the 80960MC processor activates the 
INTA signal for the first time. When the 80960MC processor receives an interrupt re quest, t h e CPU 
completes the current transaction (or comes to some interruptible po int), an d asserts INTA. INTA 
remains valid through the T a , T d , and T w states. The first assertion of INTA triggers the M8259A to 
resolve priority among its interrupt requests. 
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Figure 3-22: Timing Diagram For Interrupt Acknowledge Transaction 

To compensate for the timing of the M8259A, the 80960MC processor i nserts five Ti states before 
asserting the INTA again to read the interrupt vector. Figure 3-22 shows READY asserted without 
a wait state during the first Interrupt Acknowledgement cycle and with one wait state during the 
second Interrupt Acknowledgement cycle. In practice, the M8259A would require about four wait 
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states in both cycles. The address during the Ta state for both interrupt acknowledge cycles is 
FFFFFFFC H . For more details, see the "M8259A Programmable Interrupt Controller" section in 
Chapter 5. 



CLK2 



RESET 



A EDGE 




Figure 3-23: RESET Timing Diagram 

The 80960MC processor services the interrupt according to its priority. If the interrupt has higher 
priority than the current activity, the 80960MC processor services it immediately. Otherwise, after 
reading the interrupt vector, the 80960MC processor posts the interrupt vector in the interrupt table. 
Typically, the 80960MC processor responds within 5 us for an interrupt with higher priority than the 
current process (assuming CLK2 at 32 MHz). If the interrupt has lower priority than the current 
activity, the interrupt is serviced when its priority is higher than the priority of the subsequent activity 
of the 80960MC processor. 

Using IAC Requests For Interrupts 

The 80960MC processor can also be interrupted by an IAC message. The 80960MC processor can 
send IAC messages to itself by using one of the Synchronous Move instructions. Because this 
message does not utilize the L-bus when sent to the same processor, no special hardware is required. 
More details on IAC messages are provided in the 80960MC Programmer' s Reference Manual 

Synchronization 



The INTq/IAC, INT p INT/INTR, and INT 3 input signals can be either synchronous or asynchronous 
to the syst em clock (CLK2). To prop erly preset the interrupt signals for synchronous operation, 
INTq/IAC, INTj , INT 2 /INTR, and INT 3 must be deasserted for at least one processor clock cycle and 
asserted for at least one processor clock cycle. These signals may be deasserted and asserted 
individually. 



If the interrupt signals are asynchronous to CLK2, the 80960MC processor internally synchronizes 
them. For the CPU to recognize the asynchronous interrupt input signals, they must be preset by 
deasserting them for at least two processor clock cycles, and then asserting them for at least two 
processor clock cycles. These signals may be deasserted and asserted individually. 
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RESET AND INITIALIZATION 

The system RESET signal provides an orderly way to start or restart the 80960MC processor. When 
the 80960MC processor detects the low-to-high transition of RESET, it terminates all external 
activities and places the output pins in the high impedance state or deasserted condition. When the 
RESET signal falls low again, the 80960MC processor begins the initialization process and later 
starts fetching instructions from a specific address. 

RESET Timing Requirements 

To properly reset the 80960MC processor to a known state, the low-to-high transition of RESET must 
be asserted relative to any rising edge of CLK2 and remain asserted for at least 41 CLK2 cycles, as 
shown in Figure 3-23. RESET must be deasserted after the rising edge of CLK2, but prior to the next 
rising edge of CLK2. This establishes the next rising edge of CLK2 as edge A (the start of a bus cycle). 
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Figure 3-24: Asychronous RESET Circuit 
RESET Timing Generation 

The RESET input signal to the 80960MC processor can easily be generated by implementing a 
synchronization circuit comprised of two D-type flip-flops, as shown in Figure 3-24. 
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Figure 3-25: Diagram for RESET Timing Generation 
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Figure 3-26: Synchronous RESET Circuit 



The user RESETsignal is synchronized with the rising edge of the CL K signal by applying CLK to 
the clock input of both flip-flops. To protect against a metastable user RESET signal, the output of 
the first flip-flop, SYNC, i s applied t;o the input of the second flip-flop. The output of the second flip- 
flop r esults in a processor RESET signal. The timing diagram for thes e sign als is shown in Figure 
3-25. CLK or CLK2 can be used instead of CLK in Figure 3-24. Using CLK to clock the flip-flops 
provides a CKL2 edge A corresponding to the rising edge of CLK. Although the system architecture 
permits CLK2 and CLK to rise together at edge A, the convention used throughout this document 
is as shown in Figure 3-25. 



This preceding circuit assumed an asynchronous user RESET signal. If the user RESET signal is 
synchronous with the CLK signal, it can be implemented as shown in Figure 3-26. In this case, 
however, the output from the first flip-flop is used to generate the processor RESET signal rather than 
being routed to the input of the second flip-flop. 

Initialization 

The initialization sequence of events is shown in Figure 3-27. When RESET is deasse rted after a 
minimum of 41 CLK2 cycles, several actions take place: two input pins are sampled, the FAILURE 
output signal (see next section for the pin description) is asserted, and the self-test is performed. 

When RESET is deasserted, the 80960MC processor samples the signals r esiding on the INT^IAC 
and the BAD AC input pins (see the next section for the pin description of BAD AC). At this time, 
these pins are interpreted as the Local Process or Nu mber (LPN) and Startup (STARTUP) signals, 
respectively. A high voltage level input at the INT pin defines the 80960MC as the Primary Bus 
Master (PBM), Local Processor Number Zero (LPNO); a low voltage level defines the 80960MC as 
the Secondary Bus Master (SBM), Local Processor Number One (LPN1). During initialization of 
a uni-processor system, the 80960MC should always be assigned as the Primary Bus Master. The 
STARTUP input pin indicates whether the 80960MC processor performs initialization (high voltage 
level) or not (low voltage level). The STARTUP signal is used to allow one or more processors to 
perform the active initialization. The RESET signal timing relationships are shown in Figure 3-28. 



Besides sampling the two input pins, the 8096 0MC proce ssor asserts the FAILURE output signal a 
few cycles after RESET is deasserted. The FAILURE signal r emains ass erted while the CPU 
performs the self-test. If a failure is detected during the self-test, FAILURE remains asserted and 
the CPU enters the stopped state where the processor does nothing. At this time all outputs from the 
80960MC will be disa bled (high- impedance or deasserted). If the self-test completes successfully, 
the CPU deasserts the FAILURE signal. 
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Figure 3-27: Initialization Flow Chart 
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Figure 3-28: RESET Signal Timing Relationship 

An 80960MC processor that is designated as the initialization processor proceeds by doing a 
checksum test of eight words fetched from memory at physical address 0000 000H to en sure that 
the memory and L-bus are operating properly. If the checksum is incorrect, the FAILURE signal is 
re-asserted and the 80960MC processor enters the stopped state. After a successful checksum test, 
the 80960MC processor uses some of the previously fetched words as addresses to initial data 
structures. Complete details are provided in the 80960MC Programmer' s Reference Manual. 

Just prior to executing the first instruction, the 80960MC processor clears any latched interrupt 
signals. 

ERROR SIGNALS 



The 80960MC processor incorporates an input si gnal (BAD AC) for notification of an error condition 
in the system, and provides an output signal (FAILURE) for notification of an error within the 
processor. 



BADAC 



When asserted, the Bad Access input signal indicates to the processor that 
an unrec overable error occurred during the current data transfer. If, 
however, BADAC was asserted after a Synchronous Move or Synchronous 
Lo ad instruct ion, the error is recoverable. The 80960MC processor samples 
the BAD AC input signal during the cycle following the one when the last 
READY is asserted. 



FAILURE 



The Failure signal indicates that an e rror occurre d during initialization. The 
80960MC processor always asserts FAILURE after t he activatio n of the 
RESET signal. If a failure is detected during a self-test, FAILURE remains 



3-32 



iny 



THE 80960MC MICROPROCESSOR AND THE LOCAL BUS 



asserted. Otherwise, the processor deasserts FAILURE after a successful 
self-test is performed. If the in itial memo ry checksum is incorrect, the 
initializat ion process re-asserts FAILURE a second time, and keeps it 
asserted. FAILURE is an open drain output signal. 



SUMMARY 



The L-bus is a high speed 32-bit multiplexed bus with burst-transfer capability and is designed to 
operate with the high performance 80960MC processor. The L-bus consists of two signal groups: 
address/data, and control. These signal groups are utilized by the 80960MC processor to perform 
read, write, and burst transactions. 

The arbitration, interrupt, and reset operations are related to the L-bus transactions. The arbitration 
operation transfers control of the L-bus to another bus master. Three methods are available to handle 
interrupts: by in voking the on-chip interrupt controller, by employing an external interrupt controller 
using the INTR/INTA signals, or by using an IAC message. The reset function sets the 80960MC 
processor to a known internal state after it successfully completes the self-test. These operations offer 
power and flexibility to hardware system design using the 80960MC processor. 

This chapter focused on the L-bus and its relationship with the 80960MC processor. The next two 
chapters develop guidelines on interfacing memory and peripheral devices into the L-bus hardware 
system. 
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CHAPTER 4 
MEMORY INTERFACE 

The high-speed L-bus architecture has many features that enhance high-performance designs. In 
particular, the burst-transfer feature allows up to four successive 32-bit data word transfers at a 
maximum rate of one word every processor clock cycle. This chapter outlines approaches for 
memory designs that use these features, describes memory design considerations, analyzes the 
timing, and lists a number of useful examples. The concepts illustrated by these examples apply to 
a wide variety of memory system implementations. 

BASIC MEMORY INTERFACE 

Figure 4-1 shows the major logic blocks of a memory interface circuit. The data transceivers buffer 
the data to compensate for any slow devices that may be connected to the 80960MC processor. The 
address latches demultiplex the address/data signals from the 80960MC processor and latch the 
address. The address decoder selects the appropriate memory device from the latched address. To 
accommodate a memory burst transaction, the burst logic decrements the word count, increments the 
local address lines 3 and 2 (LAD^nd LAD 2 ), and generates a CYCLE-IN-PROGRESS signal. The 
timing control generates a READY signal and other specific signals required by a particular memory 
device. The byte enable latch stores the byte enable signals. 

Although not part of the basic memory interface, the DRAM controller, SRAM interface, DRAM, 
SRAM, and EPROM are included in Figure 4- 1 for completeness. In a hardware system the DRAM, 
SRAM, and EPROM are typically located in separate subsystems. 

Although the memory interface circuit can be designed using programmable logic, gate arrays, or 
other custom logic, the examples use standard components wherever possible to illustrate the design 
concepts. 

Data Transceivers 

Standard 8-bit transceivers can be used to provide isolation and additional drive capability for the 
L-bus. Transceivers can be used to prevent bus contention that can occur if some memories are slow 
to remove data from the L-bus after a read operation. For example, if a write operation follows a read 
operation, the 80960MC processor may drive the L-bus before a slow device has removed its output 
data, potentially causing a current spike on the power and ground lines. Transceivers, however, can 
be omitted if the data float time of the device is short enough and the load does not exceed the 
80960MC device specifications. 

The data transceivers can be con trolled by two signals from the 80960MC processor: data tr ansmit/ 
receive (DT/R) and data enable (DEN). DT/R indicates the direction of data flow and DEN enables 
the transceivers. 
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Address Latch/Demultiplexer 

Conventional transparent latches can be used to demultiplex the address/data lines of the 80960MC 
pro cesso r and to hold the address constant duri ng the memory operation. The latch is controll ed by 
the ALE signal from the 80960MC processor. ALE passes through an inv erter , so that when ALE 
goes low, the address flows through the latch. The low-to-high transition of ALE can be used to latch 
the address. The output enable of the latch can be tied to ground. The lower four address lines (LAD 3 - 
LAD ) are latched by the burst logic. 

Address Decoder 

The 80960MC processor accesses both memory and I/O devices by supplying a 32-bit address and 
a read/write command. The address decoder determines which particular memory or I/O device is 
selected by decoding the address lines. The following discussion focuses on memory selection, and 
the "Address Decoder" section in Chapter 5 discusses I/O device selection using memory-mapped 
I/O techniques. 

The memory address can be divided into regions where one region can apply to EPROM or ROM, 
another to RAM, and another to the I/O registers. In a 80960MC-based system the ROM address 
space is likely to start at address 0000 0000H because the CPU begins execution at this address. The 
RAM or I/O regions can start at any other address in the 4G-byte address range except for addresses 
FF000000H through FFFFFFFFH, which the 80960MC processor reserves for inter-agent commu- 
nication. 

Because of the large address range of the 80960MC processor, the address can be divided into word 
address bits and chip select bits. Typically the higher-order address bits are decoded to generate the 
selection signal for ROM, RAM, or I/O devices. 

The address decoder can be located either before or after the address latches. Usually, it is placed 
after the latches, so that the chip-select signal does not need to be latched. Figure 4-1 shows the 
address decoder placed after the address latches. 

Burst Logic 

To enhance system performance, the 80960MC processor performs burst transactions that transfer 
up to four data words at a maximum rate of one word every clock cycle. A DRAM controller can 
be designed that takes advantage of the burst-transfer capability by using the static column mode or 
nibble mode features of the DRAM (see the "DRAM Controller" section of this chapter. This DRAM 
controller requires a signal, called CYCLE-IN-PROGRESS, to identify the start and end of a memory 
cycle. The burst logic generates the CYCLE-IN-PROGRESS signal. 



Figure 4-2 shows the flow chart for the burst logic. If ADS is low and DEN is high, then the burst 
logic latches LAD 3 through LAD , and asserts the CYCLE-IN-PROGRESS signal. The burst logic 
checks the SIZE signals (LADj and LAD ). If the value of the SIZE signals equal zero, then the burst 
logic runs one memory cycle, and terminates the CYCLE-IN-PROGRESS signal. If the value of the 
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SIZE signals do not equal zero, the burst logic runs one memory cycle, increments the latched 
address's A 2 and A 3 to point to the next 32-bit word, and decrements the SIZE value. When this is 
finished, the burst logic checks the value of the SIZE signals again. 
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Figure 4-2 Burst Logic Flow Chart 

The burst logic can be used with EPROM, SRAM, DRAM memories. However, it cannot be used 
in the DRAM static column or nibble modes, because they do not support burst transactions/Because 
the 80960MC processor ensures that a burst transaction cannot exceed four words or cross a 16-byte 
boundary, incrementing A 3 and A 2 after a single data word transfer makes the burst transfer 
transparent to the memory devices. 
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Timing Control Logic 

The timing control logic accommodates memory devices that cannot transfer information at the 
maximum bus rate by inserting wait states until the data becomes available. The timing control logic 
consists of a counter and timing logic, as shown in Figure 4-3. The counter produces a 4-bit binary 
count. The cou nt begins when the CYCLE-IN-PRdGRESS sig nal is asserted . The t iming logic 
asserts READY at the appr opriate tim e based upon the count, the EPR OM-CS, an d the SRAM-CS 
signals. For a burst transfer, READY resets the counter to properly time a READY signal for the next 
data transfer. When CYCLE-IN-PROGRESS is deasserted, the clock counting is terminated. 



Because the timing of DRAM is more complicated, the DRAM controller generates a D RAM-RDY 
signal to the timing control logic. I n addition, th e cl ock count, t he W/R command, and SRAM-CS 
signal can also be used to generate SRAM- WE and SRAM-OE signals. 
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Figure 4-3: Memory Timing Control Block Diagram 



Byte Enable Latch 



The byte enable latch holds the byte enable signals constant until the DRAM controller or SRAM 
interface uses the signals. As mentioned in the "L-Bus Signal Groups" section in Chapter 3, the byte 
enable signals specify which bytes (up to four) on the 32-bit data bus are transferred during the data 
cycle. Each individual byte enable signal selects eight data lines as shown in Table 4-1. 

The byte enable signals are valid from the 80960MC processor before data is transferred. These 
signals are asserted during the address cycle for the first data word transfer; they are asserted again 
during the first data cycle for the second word transfer; the second data cycle for the third word 
transfer; and the third data cycle for the fourth word transfer. 
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The ALE signal can be used to latch the first byte enable signals. READY can be used with the A 
edge of CLK or CLK2 to latch the other byte enable signals for each word. 

Table 4-1 : Byte Enable Signal Decoding 



Byte Enable Signal 


LAD Line Selection 


B5,' 


LAD 7 -LAD 


BE, 


LAD 15 -LAD 8 


H* 


LAD 23 -LAD 16 


BE 3 


LAD 31 -LAD 24 



SRAM INTERFACE 

The basic memory interface can be used in conjunction with the SRAM interface to read and write 
to SRAM. This section describes the SRAM interface and examines the timing. 

SRAM Interface Logic 



The SRAM interface logic uses the latched byte enable signa ls, the SR AM-OE, an d the SRAM- WE 
signals to gene rate four ou tput ena ble signals (S RAM-OE^ through SRAM-OE ) and four write 
enable signals (SRAM-WE 3 through SRAM-WE ), as shown in Figure 4-4. These signals allow the 
80960MC processor to write to the data byte that is specified by the byte enable signals. SRAMs with 
separate OE and CS signals require only one OE signal per bank since the 80960MC ignores 
unrequested bytes in read operations. 



SRAM Timing Considerations 

This section analyzes the critical timing paths of the SRAM control signals. From the critical path, 
the timing equations can be derived to determine the memory access time for no wait state operation. 

When evaluating critical timing paths, the timing calculations should use worst-case data sheet 
parametric specifications, rather than typical specifications. By using worst-case timing values, 
reliable operation is assured over all variations in temperature, voltage, and individual device 
characteristics. These timing values are determined by assuming the maximum propagation delay to 
latch an address, select a memory device, and pass through data buffers and transceivers. 



Figure 4-5 shows the critical timing path for a one- word SRAM read operation. The diagram consists 
of three time periods: the address setup period (T addrset ), the memory response period (T mem ), and the 
data return period (T dataset ). Note that the timing for the read command and output control signals does 
not enter into the critical timing path. 
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Figure 4-4: Logic Diagram for SRAM Interface 

During the T addrset period , the 8Q96QMC processor outputs a valid addres s that is latc hed on the low- 
to-high transition of the ALE signal. The address decoder generates the SRAM-CS signal from the 
latched address and the Timing Control/SRAM Interface logic subsequently generates the OE 
signals. During the T mem period the SRAM responds to the commands and signals and retrieves the 
data. The access time of the memory determines the duration of the T period. T can be varied 

J mem r mem 

in increments of clock cycles by delaying the READY signal. 

The data must be available at the address/data pins of the CPU before the end of the data state. The 
T dataset period must take into account the setup time requirement of the 80960MC processor and the 
throughput delay of a data transceiver. 
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Figure 4-5: Critical Timing Path for SRAM Read Operation 

For a no wait state operation, the data transfer word must be completed in two system clock (CLK) 
cycles. The minimum time period for a no wait state operation (T mem _ no _ wait ) can be determined by 
using equation 1. 



T ., = 2CLK - T 

mem-no-wait 



where: 



mem-no-wait 

2CLK 
T 

addrset 



-T 

addrset dataset 

= Memory access time for no wait state operation 

- Two system clock (CLK) cycles 

= Maximum delay to valid address 
+ Maximum throughput delay of address latch 
+ Maximum delay to generate chip select 
+ Maximum delay to generate SRAM-OE n 

= Maximum delay through data transceiver 
+ Maximum data setup time of CPU 



(1) 



A similar analysis can be done for burst transactions. Equation 1 can be used to determine the access 
time for no wait state operation of the first word. For subsequent words, equation 2 can be used. In 
this equation, the address setup time is replaced by delay in the burst logic to change the address 
(T burst ). In this case, the data transfer of each subsequent word must be completed in one system clock 
(CLK) cycle (no address state). The minimum access time for a no wait state operation (T mem _ no _ wait ) 
can be determined by using the lesser value of equation 1 or equation 2. 
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T , = CLK - T, 

mem-no-wait ' 



where: 



mem-no-wait 



CLK 



burst 



burst dataset 

= Memory access time for no wait state operation 

= One system clock (CLK) cycles 

= Maximum delay to change the address 

= Maximum delay through data transceiver 
+ Maximum data setup time of CPU 



(2) 



The memory access time can be extended by delaying the READY signal and adding wait states. 

The timing analysis described for a SRAM read operation can be used for EPROM timings. If 
EPROMs are only used to store initialization programs, they are seldom accessed compared to 
memory devices used to store program data or instructions. Consequently, the addition of wait states 
during the read cycle does not affect overall system performance. 

Figure 4-6 shows the critical timing path for an SRAM write operation. The diagram consists of two 
time periods: the address setup period (T addrset ) and the memory response period (T mem ). 
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Figure 4-6: Critical Timing Path for SRAM Write Transaction 

During the T addrset pe riod, t he 80960MC processor outputs a vali d address th at is latched on the low- 
to-high transition of ALE. The address decoder generates the SRAM-CS signal from the latched 
address and the Timing Control/SRAM Interface logic subsequently generates the Write Enable 
(WE) signals. 
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During the T mem period the SRAM responds to the commands and writes the data. The access time 
of the memory determines the duration of the T period. T can be varied in increments of clock 

J „ mem r mem 

cycles by delaying the READY signal. 

Two timing paths should be considered during the T mem period: the path where data is supplied to the 
memory, and the path that monitors the memory write cycle time. The first path takes into account 
the time for the 80960MC processor to generate valid data, the throughput delay of a data transceiver, 
and the data setup and hold time requirements of the memory devices. The second path is the memory 
write cycle specification. The longer of the two paths is the critical timing path. 

By examining the timing path required to write to the SRAM, equation 3 can be derived which 
determines SRAM write cycle time for ho wait state operation. The memory cycle time is determined 
by the lesser value of equation 1 (read) or equation 3 (write). 

* mem-no-wait = 2CLK " * addrset («3) 

where: T ., => Maximum delay to valid data 

mem-no-wait J 

+ Maximum throughput delay of data transceiver 
+ Maximum data setup and hold times of memory 

2CLK = Two system clock (CLK) cycles 

T addrset = Maximum delay to valid address 

+ Maximum throughput delay of address latch 
+ Maximum delay to generate chip select 
+ Maximum delay to generate write enable 

The mem ory access time for either memory reads or memory writes can be extended by delaying the 
READY signal and generating wait states. 

DRAM CONTROLLER 

This section provides design guidel ines f or a DRAM controller. Many DRA Ms offer static column 
mode and Column Address Strobe (CAS) before Row Address Strobe (RAS) refresh features. This 
section shows guidelines on how to use these features with the burst capability of the 80960MC 
processor to significantly enhance system throughput. 

The DRAM controller multiplexes the address into a row and column address, performs the refresh 
operation, arbitrates between a refresh request and memory request, and generates the necessary 
control signals for the DRAM. To implement these functions, the memory controller uses an address 
multiplexer, arbiter, refresh interval timer, and DRAM timing and control as shown is Figure 4-7. 

A standard VLSI DRAM controller can be used, but it typically degrades system performance. 
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Address Multiplexer 

The address multiplexer divides the DRAM address into a row and column address. The prope r 
selection of a row or column address is accomplished by the row/column select signal (ROW/COL) 
from the DRAM timing and control circuit. 

Refresh Interval Timer 



The refresh interval timer periodically generates a refresh request (REF-REQ) by counting enough 
bus cycles to equal the refresh interval period. Since a refresh request is processed after a completed 
operation, the refresh period must take into account the time required to perform a bus operation, as 
well as the DRAM refresh specification. For example, a lM-bit DRAM that requires 512 refresh 
cycles within 8 ms needs a refresh cycle every 15.6 us. To meet the pRAM specification, the refresh 
interval timer must generate a refresh request in less than 1 5 .6 us to compensate for any required time 
to complete the operation with wait states. 
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Figure 4-7: DRAM Controller Block Diagram 
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After the REF-REQ signal is generated, the arbiter sends a refresh acknowledge si gnal REF-A CK 
back to the interval timer to assure that refresh occurred before generating another REF-REQ. 

Arbiter 

DRAM controller uses an arbiter to decide whether a memory cycle or refresh cycle is performed. 
In a synchronous design, arbitration is easily performed because memory and refresh cycle requests 
never occur at or near the same time. 

The arbiter monitors memory cycle req uests and re fresh requests. The arbiter detect s a DRAM 
memory request by decoding two signals: DRAM-CS and CYCLE-IN-PROGRESS. The REF-REQ 
signal indicates that a refresh cycle must be performed. The arbite r arbitrates between a memory 
cycle or refresh cycle and gen erate s a Memory/Refresh (MEM/REF) signal. The DRAM timing and 
control block uses the MEM/REF signal to start the generation of the control signals. 



When a refresh cycle is performed, the arbiter sends a REF-ACK signal to the refresh timer, which 
uses this signal to begin another count cycle. 

DRAM Timing and Control 

The DRAM timing and control circuit is the final logic block and core of the DRAM controller. The 
functions of this circuit include the following: 



• Generating the DRAM control signals (RAS, CAS, and WE) with the proper timing relation- 
ships during system operation 



• Generating the DRAM-RDY signal 

• Performing the refresh function by asserting CAS before RAS 

• Performing several warm-up cycles required by the DRAM when power is first applied. 

The DRAM timing and control logic can be designed to take advantage of the burst-transfer 
capability of the 80960MC processor by implementing static column mode or nibble mode. With 
nibble mode, a multiplexed address is ap plied to the DRAM, and up to four bits of data are quickly 
transferred by successively toggling th e CA S pulse. The DRAM timing and control logic can be 
designed to p rovide the successive CAS pulses by using the CYCLE-IN-PROGRESS and 
DRAM-RDY signals. 

Static column mode can also be used to take advantage of the burst capability of the DRAM. Static 
column mode allows fast access to the bits located in the selected row of the DRAM simply by 
changing the column address after the first access. 

Figure 4-8 shows a flow chart for the DRAM timing and control logic using static column mode. The 
DRAM timing and control circuit receives a refresh request or a memory request on the MEM/REF 
and CYCLE-IN-PROGRESS input signals. For a memory request, theDRAM timing and control 
determines whether a read or a write operation is desired from the W/R signal from the 80960MC 
processor. 
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SAMPLE MEM/REF AND 

CYCLE-IN-PROGRESS 

INPUTS 



MEMORY CYCLE 
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Figure 4-8: Flow Chart For DRAM Timing and Control Logic 
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For a read o peration, the DRAM timing and contr ol log ic performs sever al fun ctions: it brings 
ROW/COL high to select a ro w addres s; it a sserts RAS ; it brings ROW/COL low to select the 
column address; i t asserts CAS ^ through C A~S n (derived fr om the four latched byte enable signals); 
and it generates a DRAM-RDY signal. The DRAM-RDY signal causes the burst logic to increment 
the address and the 80960MC processor to read the data word. 

After completing these functions the DRAM timing and control logic samples the CYCLE-IN- 
PROGRESS to determine wheth er to transfer another data word. If so, the DRAM timing and control 
logic maintai ns the ROW/COL si gnal low to select the new column address, deasserts and asserts 
CAS^ through CAS to observe the CAS precharge specification of the DRAM, and generates another 
DRAM-RDY, The DRAM timing and control logic repeats the proc edure until all the data words 
are transferred. Then the DRAM timing and control logic deasserts RAS . 

For a write oper ation , the DRAM ti ming a nd control logic performs similar funct ions o n the first 
word: it ass erts WE; it brings ROW/COL high to select a row address; it asserts RAS ; it brings 
ROW/COL low to select the column address; it asserts CAS ^ through CAS n (derived from the four 
latched byte enable signals); and it generated a DRAM-RDY signal. The DRAM-RDY signal causes 
the burst logic to increment the address and informs the 80960MC processor by asserting READY 
that the data word was written. 

After completing these functions the DRAM timing and control logic samples the CYCLE-IN- 
PROGRESS to determine whether the S0960MC wan ts to transfer another data word. If so, the 
DRAM timing and control l ogic maintain s th£ ROW/COL si gnal l ow to select the new column 
address, deasserts and asserts C AS^ through CA S ft to observe the CAS precharge specification of the 
DRAM, and generates another DRAM-RDY. The DRAM timing and control logic repeats the 
proce dure until all the data words are transferred. Then the DRAM timing and control logic deasserts 
RAS„. 



Although only one RAS sign al is re quired, four CAS signals (CAS3-CAS ) are generated to enable 
each byte of the L-bus. These CAS signals are generated by the byt e enab le decoder and correspond 
to the byte enable signals of the 80960MC processor. For example, CAS , which is mapped directly 
from BE , selects the least-significant data byte (LAD 7 -LAD ). 



A single WE control signal and four CAS signals ensure that only those DRAM bytes selected for 
a write cycle are enabled. All other data byte s maintain their output s in t he high-impedance state. 
A co mmon design error is to use a single CAS control signal and four WE control signals, using the 
WE signals to write the DRAM bytes selectively in write cycles that use fewer than 32 bits. Although 
the selected bytes are written correctly, the unselected bytes are enabled for a read cycle. These bytes 
output their data to the unselected bytes of the data bus while the data transceivers output data to every 
bit of the data bus. When the two devices simultaneously output data to the same bus, bus contention 
occurs. 



The re fresh function can be performed by asserting the CAS signal before asserting RAS. The CAS 
befor e RAS refresh feature eliminates the need for an ex ternal refresh address counter. When the 
CAS pulse is activated prior to the assertion of the RAS pulse, the DRAM automatically performs 
a refresh cycle on one row by employing an on-chip address counter. Upon completion of the refresh 
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cycle, the address counter is automatically incremented. The ME M/REF signal from the arbiter can 
be used by the DRAM timing and control logic block to initiate a CAS before RAS refresh cycle. 



Besides generating the RAS, CAS, and WE signals, the DRAM timing and control logic generates 
a number of warm-up cycles for the DRAM after reset by issuing several refresh requests. 



Timing Considerations For The DRAM Controller 

Figure 4-9 shows a typical example of a timing diagram for a two- word read transaction that uses 
static column mode; similarly, Figure 4- 1 is a typical example for a two-word write transaction. The 
example assumes a memory access time that requires two wait states (T w ) for the initial data word 
and one wait for the second data word. 
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Figure 4-9: Timing Diagram for Two-word Read Transaction 

The critical timing areas for both read and write transactions are noted by circled numbers in the 
diagrams, which are enumerated below. 

1 . The delay for the CPU to generate a valid address. 

2. The delay for the DRAM timing and control logic to generate the CYCLE-IN-PROGRESS 
signal. 
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3. The delay to generate the DRAM row address. This time includes the address latch throughput 
delay, the multiplexer throughput delay, and the address driver delay. 



4. The delay to generate RAS, which includes the delay to generate the DRAM-CS signal. 



5. The row address hold time after the high-to-low transition of RAS. 

6. The time required to generate the multiplexer control signal (ROW/COL) after the row address 
hold time is satisfied. 

7. The time required to switch from a row to column address plus any driver delays. 

8. The delay to generate and drive the CAS signals. 

9. For a read transaction, the throughput delay of the data transceivers. For a write transaction, the 
delay by the CPU to generate valid data. 

10. For a read transaction, the data setup time of the CPU. For a write transaction, the throughput 
delay 6f the data transceivers. 

11. The time required to increment and drive the column address. 



1 2. For a write transaction only, the delay time to bring CAS high (terminate the CAS pulse for the 
first data byte), to precharge the CAS pulse (required by the DRAM), and to assert CAS again. 



13. The RAS precharge time, which must be satisfied before another memory cycle can begin. 



DATA FROM 
TRANSCEIVER 
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Figure 4-10: Timing Diagram for Two-word DRAM Write Transaction 
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DRAM Interleaving 

Because the DRAM consists of dynamic nodes, a row precharge time is required to recharge the 
nodes after every memory cycle. This time must be included in the timing evaluation, as noted by 
the example. To avoid the precharge time delay of the DRAM, the memory array can be arranged 
so that each subsequent memory access is most likely to be directed to a different bank. In this 
configuration, wait time between accesses is not required because while one bank of DRAMs 
performs the current access, another bank precharges and is ready to perform the next access 
immediately. 

If DRAMs are interleaved (i.e., arranged in multiple banks so that adjacent addresses are in different 
banks), the DRAM precharge time can be masked for most accesses. With two banks of DRAMs, 
one for even 32-bit addresses and one for odd 32-bit addresses, all sequential 32-bit accesses can be 
completed without waiting for the DRAM to precharge. 

Even when random accesses are made, two DRAM banks allow 50 percent of back- to-back accesses 
to be made without waiting for the DRAMs to precharge. The precharge time is also masked when 
the 80960MC processor has no bus accesses to be performed. During these idle bus cycles, the most 
recently accessed DRAM bank can precharge so that the next memory access to either bank can begin 
immediately. 

SUMMARY 

The memory interface circuit allows the 80960MC processor to communicate with the memory 
devices. The basic memory interface logic can be divided into six blocks: the data transceivers, the 
address latches, the address decoder, the burst logic, the DRAM timing and control logic, and the byte 
enable latch. The DRAM controller and SRAM interface complete the memory interface circuit. The 
DRAM controller can be designed to take advantage of the 80960MC processor's burst capability 
to enhance system performance. 

This chapter focused on the design guidelines for the memory interface design to the 80960MC 
processor. Chapter 5 develops guidelines on designing peripheral devices in the single-processor 
hardware system. 
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CHAPTER 5 
I/O INTERFACE 

The 80960MC processor supports 8, 16, 32-bit I/O devices by mapping them into its 4 G-byte 
memory address space. This chapter describes the design considerations for the interface between 
the 80960MC processor and I/O components. Several examples illustrate the design concepts. 

INTERFACING TO 8-BIT AND 16-BIT PERIPHERALS 

The 80960MC processor accesses I/O devices by using a memory-mapped address. Consequently, 
memory-type instructions can be used to perform input/output operations. For example, the 
80960MC processor's LOAD and STORE instructions can directly support 8-bit and 16-bit data 
moves to or from I/O peripherals. The instructions include those listed below. 

• Load Ordinal Byte (reads a byte) 
Load Ordinal Short (reads 16-bit data) 

• Store Ordinal Byte (writes a byte) 

• Store Ordinal Short (writes 16-bit data) 

These instructions perform the transfer on the data bits specified by the two low-order lines of the 
effective address. See the 80960MC Programmer' s Reference Manual for complete details. 

GENERAL SYSTEM INTERFACE 

In a typical 80960MC processor system design, a number of slave I/O devices can be controlled 
through a general system interface. Other I/O devices, particularly those capable of controlling the 
L-bus, can use the general system interface, but may require additional logic to isolate the bus. This 
section describes the general system interface and assumes that the 80960MC processor does not 
perform burst transactions to the I/O devices. 

Figure 5-1 shows the major logic blocks of the general system interface. Standard 8-bit data 
transceivers add drive capability, provide bus isolation, and prevent bus conflicts that may occur with 
slow I/O components. The address latch demultiplexes the address/data lines and holds the address 
stable throughout the L-bus transaction. The address decoder generates the I/O chip- select signals 
from the latched address lines. The timing control block provides the READY signal to the 
80960MC processor and the I/O read and I/O write command. 

This basic interface circuit is quite similar to the one used in the basic memory interface described 
in Chapter 4. For most systems the same data transceivers, address decoders, and address latches can 
be used to access both memory and I/O devices. The timing control logic can be implemented to 
accommodate both memory and I/O devices. 
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Figure 5-1 : Simplified I/O Interface 
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Data Transceivers 

Standard 8-bit transceivers can be used to provide isolation and additional drive capability for the L- 
bus. Transceivers prevent bus contention that can occur if some devices are slow to remove data from 
the data bus after a read cycle. For example, if an I/O write cycle follows a I/O read cycle, the 
80960MC processor may drive the L-bus before a slow device has removed its outputs from the bus, 
potentially causing a current spike. Transceivers, however, can be omitted if the data float time of 
the device is short enough and the load does not exceed the 80960MC device specifications. 

The data transceiver can be contr olled b y two_signals from the 80960MC processor: D ata Tr ansmit/ 
Receive (DT/R) and Data Enable (DEN). DT/R indicates the direction of data flow and DEN enables 
the transceivers. 

Address Latch/Demultiplexer 

Standard transparent latches can be us ed to demultiplex the address/data lines of th e 809 60MC 
processor. The latch is controlled by the A LE si gnal from the 80960MC processor. The ALE signal 
passes through an inverte r, suc h that when ALE goes low, the address flows through the latch. The 
low-to-high transition of ALE can be used to latch the address. 

If only slave-type peripherals are used in a system, the output enable of the latches can always remain 
active by connecting it to ground. For systems with DMA devices, the output enable can be used to 
permit the DMA device to drive a common address bus. 

Address Decoder 

The address decoder determines which particular I/O device is selected by decoding the address. The 
I/O address can be any address in the 4 Gbyte address range except for the upper 1 6 Mbytes (addresses 
FF000000 H through FFFFFFFF H ), which the 80960MC processor reserves for inter-agent commu- 
nication and internal I/O. Typically, a small range of address bits are reserved for accessing I/O 
devices by defining certain higher-order address bits as an I/O access. 

As an example, consider a 32-bit address: A^ through A 15 could indicate an I/O access when A 31 is 
set to zero, and A 30 -A 15 are set to one; A 14 through A 5 could then be used to specify a particular I/O 
device; and A 4 through A 2 can be used to access up to 8 registers of the I/O component. A { and A 
are not used by the I/O device. This particular scheme selects up to 1 ,024 devices, while using only 
32K bytes of the available 4 Gbytes of address space. 

The address decoder can be located either before or after the address latches. Usually, it is placed 
after the latches, so that the chip-select signal does not need an additional latch. 

Timing Control Logic 

The timing control logic accommodates I/O devices that cannot transfer information at the maximum 
bus rate by inserting Wait States until the data becomes available. The timing control logic consists 
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of a counter and timing logic, as shown in Figure 5-2. The counter produ ces a 4-b it binary count. 
The co unt is star ted at the beginning of the operation (determi ned by ADS and DEN) and is sto pped 
by th e READY signal. The timing logic as serts the READY signal, the I/O write command (I/O- 
WR), a nd the I/O read command (I/O-RD) based upon the clock count, the I/O chip select signal 
(I/O-CS), and the W/R command. 
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Figure 5-2: I/O Timing Control Block Diagram 



For many peripherals, the timing logic can be programmed to assert READY at the appropriate count 
for the selected device. Specific I/ O chip select signals can be used to indicate how many clock cycles 
to wait before asserting READY. 



For some I/O peripherals, parti cularly bu s masters, READY cannot be determined by counting clock 
cycles. For these I/O devices, READY can be supplied by the device and passed on to 80960MC 
processor. 



The timing control block can assert the I/O-RD or I/O- WR signal for I/O devices based upon the clock 
count. The timing for these signals can be selected for the slowest device to simplify the logic circuit 
or can be customized for each individual peripheral device to maximize performance. 



I/O INTERFACE DESIGN EXAMPLES 

The general system interface shown in Figure 5-1 can be used to connect the 80960MC processor 
to many slave peripherals. The following list includes some common peripherals compatible with 
this interface: 

• M8259A Programmable Interrupt Controller 

• M8253, M8254 Programmable Interval Timer 

• M82510, Asynchronous Serial Controller 
M8274 Multi-Protocol Serial Controller 
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• M8255 Programmable Peripheral Interface 

• 82586 LAN Coprocessor (not offered in a MIL-STD-883C version) 

• M82786 Graphics Coprocessor 

This section provides guidelines and design considerations for interfacing the 80960MC processor 
to different types of I/O configurations. Specifically, three design examples are examined. The 
M8259 A design example shows how to interface the 80960MC processor to a slave-type peripheral 
device. The 82586 design example shows how a 16-bit bus master reads and writes to the 8096QMC 
processor's system memory. The M82786 design example shows how the 80960MC processor can 
read or write to graphics memory using a 16-bit data bus. 

M8259A Programmable Interrupt Controller 

The M8259A Programmable Interrupt Controller is designed for use in interrupt-driven microcom- 
puter systems, where it manages up to eight independent interrupts. The M8259A handles interrupt 
priority resolution and returns an 8-bit vector to the 80960MC processor during an interrupt- 
acknowledge cycle. Intel Application Note AP-59 contains detailed information on configurations 
oftheM8259A. 

Interface 

Figure 5-3 shows the connection of the 80960MC processor to a single M8259A Interrupt Controller. 
This circuit consists of the general system interface plus a bidirectional buffer. The example assumes 
that several interrupt requests occur at the same time so that priority resolution is required. 

The data lines from the M8259A are not directly aligned to the 80960MC processor because of the 
difference in priority resolution between the devices. Although both devices use an 8-bit interrupt 
vector, the 80960MC processor implicitly defines the priority by dividing the interrupt vector by 
eight. The M8259A defines the priority in the lower three bits of the interrupt vector. Furthermore, 
the highest priority vector of the 80960MC processor has a value of 3 1 in the upper five bits of the 
interrupt vector. Whereas, the highest priority interrupt of the M8259A has a value of in the lower 
three bits of the interrupt vector. 

To resolve the priority difference, the interrupt vector from the M8259A can be inverted and rotated 
left by three bits as shown by the data alignment between the 80960MC processor and M8259A in 
Figure 5-3. Rotating the data bits in this manner provides two advantages: the interrupt table for the 
M8259A can be located by contiguous addresses, and the upper two most significant bits of the 
interrupt vector remain free to group interrupt vectors if additional M8259As are needed. 

Care must be exercised, however, when programming the registers of the M8259A. For example, 
assume that the second initialization command word (ICW2 register) of the M8259 A requires a data 
byte value of 0001 1 1 l"l B .'To transfer the correct information, the 80960MC processor needs to write 
a data word with the value of 000001 1 1 B because this word is rotated left three places and inverted. 
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Operation 

The M8259A starts the interrupt cycle by generating an interrupt request (INT) to the 80960MC 
processor, which receives the signal at the INTR input pin. This assumes the Interrupt Control 
register of the 80960MC processor is set to accommodate an external interrupt controller. 



When the 809 60MC processor comes to a breakpoint in its execution, it asserts the INTA signal 
twice. The first INTA signal acknowledges the interrupt reques t and causes the M8 259 A to prior itize 
the interrupt requests it received up to this poin t. The IN TA, together with the M8259A-CS, are 
applied to the timing control logic to generate a READY signal. 



The 8096Q MC proce ssor automatically asserts th e seco nd INTA signal five clock cycles after the 
assertion of READ Y. After the second assertion of INTA, the 80960MC processor reads the interrupt 
vector from the M8259A. 

The bidirectional buffer inverts and passes the 8-bit vector to the 80960MC processo r with the 
appropriate lines rearranged. The output enable signal for the data buffer is controlled by INTA for 
this oper ation. After the data transfer is completed, the timing control circuit generates a second 
READY signal to terminate the interrupt acknowledge cycle. 

The same circuitry can be used to read or write to the M8259A registers. In this case, the 80960MC 
processor selects the M8259 A through a memory-mapped address. Local address line 2 (A 2 ) selects 
one of two internal registers of the M8259A. The I/O read or I/O write command is generated by the 
timing control circuit. The data passes through the bidirectional data buffer to or from the selected 
register of the M8259 A. 

The direction of data flow through the buffer is controlled by three logic gates shown in Figure 
5-3. For an I/O write operation, the I/O Write command and M8259A-CS signal control the output 
enable signal of the bidirectional buffer. Similarly, for a read operation, the I/O Read command and 
the M8259A-CS signal control t he output enable signal of the buffer. After the data is transferred, 
the timing control circuit asserts READY. 

82586 Local Area Network Coprocessor Example 

The 82586 (not offered in a MIL-STD-883C version) is an intelligent, high-performance commu- 
nications controller designed to perform most tasks required for controlling access to a local area 
network (LAN), such as Ethernet or Starlan. In many applications, the 82586 is the communication 
manager for a station connected to a LAN controller. Such a station usually includes a host CPU, 
shared memory, a Serial Interface Unit, a transceiver, and LAN controller link, as shown in Figure 
5-4. The 82586 performs all functions associated with data transfer between the shared memory and 
the LAN link, including: 

• Framing 

• Link management 

• Address filtering 
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• Error detection 

• Data encoding 

• Network management 

• Direct memory access 

• Buffer chaining 

• High-level (user) command interpretation 
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Figure 5-4: Lan Station 
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The 82586 has two interfaces: a 16-bit bus interface and a network interface to the Serial Interface 
Unit. The bus interface is described here. For detailed information on using the 82586, refer to the 
Local Area Networking Component User's Manual. 
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Figure 5-5: Block Diagram For LAN Controller Interface 
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Interface 

There are several ways to design an interface between the 82586 and the 80960MC processor. The 
chosen design example shows how to interface the 82586 using a shared bus. In this example, the 
82586 operates in minimum mode at one-half the processor clock frequency. 

The primary function of the interface circuit is to allow the 82586 to read and write 16-bit data using 
the 32-bit L-bus. This is accomplished by adding the high-order address lines and translating the 1 6- 
bit data lines to the 32-bit data lines by using byte enable signals. 

Figure 5-5 shows the 82586 interface circuit, which includes the DRAM controller (see the "DRAM 
Controller" section in Chapter 4. This interface uses the general system interface circuit plus other 
logic units that specifically pertain to the 82586: the LAN data transceivers, the byte enable 
converter, and the LAN address latches. These logic blocks are highlighted by the shaded boxes. 




Figure 5-6: Byte Enable Generation Circuit 

The LAN data transceivers connect 16 data lines from the 82586 to both th e upp er and lower 16 bits 
of the L-bus. The data transfer is controlled by converting A , A v and the BHE to four byte enable 
signals as shown in Figure 5-6. Al selects between the upper and lower 16-bit data lines; \ selects 
t he low er data byte for either the upper or lower 16-bit data lines; and the byte high enable signal 
(BHE) selects the upper data byte for either the upper or lower 16-bit data lines. Data flows through 
the buffers when the appropriate byte enable signal is asserted. The direction of the data flow is 
controlled by the DT/R signal of the 82586. 



The LAN address latches are used to demultiplex AD 15 through AD . The address lines and BHE are 
latched by the ALE signal from the 82586. The upper address lines (A^ through A 16 ) are generated 
by hardware programmable DIP switches. 
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The 82586 begins operation when the Channel Attention (CA) input signal is asserted. This signal 
is generated by gating the write command and 82586 chip select signal. 
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Figure 5-7: Operational Flow Diagram for 82586 Interface 



Operation 



The interaction between the 82586 and the 80960MC processor is described below and is summa- 
rized in Figure 5-7. 

• The 80960MC processor invokes the 825 86 by supplyi ng a memor y-mapped address and a write 
command. The memory-mapped address results in a 82586-CS signal, which is gated with a 
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write command to produce the CA signal. 

The 82586 responds by generating a hold request and waits for HLDA. 

The 80960MC processor asserts HLDA, which enables the outputs of the LAN address latches 
and disables the outputs of the address latches next to the 80960MC processor. The HLDA 
signal also gives control of the L-bus to the 82586. 

After the 82586 takes control of the bus, it generates a 16-bit address (AD 15 through AD ), an 
ALE signal, and a BHE signal. The upper address lines are provided by the programmable DIP 
switches to produce an address on the L-bus. 



Aj and A (from the 82586), and BHE are decoded to generate four byte enable signals (BE 3 
through BE ). DEN enables the output of the byte enable converter. 

DT/R from the 82586 controls the direction of the data flow through the buffers. 

The read or write signal from the 82586 is applied to the DRAM controller. 

The 82586 accesses DRAM by using the DRAM controller. 



• The DRAM-RDY is asserted by the DRAM controller. This action enables the output of the 
LAN data transceiver and terminates the 82586 memory cycle. The timing control logic passes 
the DRAM-RDY signal as the READY signal to the 82586. 

• The 82586 deasserts HOLD and the 80960MC processor deasserts HLDA. The 80960MC 
processor regains control of bus. 

M82786 Graphics Coprocessor Example 

The M82786 is a high performance graphics coprocessor that provides high quality text and 
advanced display control. It provides full support for graphics primitives at up to 25 million pixels 
per second and bit-mapped text up to 25 thousand characters per second. This graphics processor 
supports advanced features such as hardware windows, zooming, panning, and scrolling. Intel 
Application Note AP-259 and Application Note AP-270 contain detailed information on 82786. 

When using the M82786, it may be necessary for the 80960MC processor to write to graphics 
memory. The interface design example illustrates how the 80960MC processor can transfer a 32- 
bit data word to the 16-bit data bus of the M82786. 

Interface 

There are several ways to design an interface between the M82786 and the 80960MC processor. In 
this example, the 80960MC processor reads or writes to graphics memory by accessing the M82786 
through the interface logic circuit. This example assumes that the M82786 operates in the slave 
mode, and that the 80960MC processor does not perform burst transfers. The 80960MC processor 
only performs burst transfers for instructions that specify accesses for more than one word or for 
instruction fetches. 
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The interface circuit translates a 32-bit data bus to a 16-bit data bus by dividing the data lines into 
the upper and lower 16 bits and sequencing the data transmission. When the 80960MC processor 
writes to graphics memory, the bidirectional transceivers sequence the lower and the upper data bits 
of the L-bus to the 16-bit data bus of the M82786. 

The process is reversed when the 80960MC processor reads from graphics memory. The bidirec- 
tional transceivers form a 32-bit data word by latching the first 16-bit data word on the lower data 
lines, routing the next 16 bits to the upper data lines, and then passing the 32-bit data word on the L- 
bus. 

Figure 5-8 shows the details of the graphics controller interface circuit. This interface uses the general 
system interface circuit plus the following logic units: the bidirectional transceivers, the data buffer 
control, the data bus controller, and the address translator. Theselogic blocks are highlighted by the 
shaded boxes. 

The bidirectional transceivers pass data to (from) a 32-bit data bus from (to) a 16-bit data bus. Data 
is sequenced through the transceivers by the control signals generated by the data buffer controller. 

The data buffer control logic generates the signals that operate and sequence the bidirectional 
transceivers. The direction signal for data flow through the transceivers is derived from the W/R 
signal of the 80960MC processor. The data buffer control lo gic ge nerates four output enable signals: 
GAB L enables the outputs on the B side for the lower 16 bits; GB A L enables the outpu ts on th e A side 
for the lower 1 6 bits; G AB H enables the outputs on the B side for the higher 1 6 bits; and GB A H enables 
the outputs on the A side for the higher 16 bits. These output enable signals are derived from the byte 
enable signals and are asserted when the slave enable signal (SEN) is activated by the M82786. 

The select lines for the bidirectional transceivers allow data to flow from either the latched data or 
the input pins. These lines, which are not shown, can be hardwired. 

The data bus controller provides the read (RD) and write (WR) commands, memory or I/O signal 
(M/IO), and a READY signal. This circuit generates two read or write commands for every 32-bit 
data transfer to or from the 80960MC processo r (one for eac h 16-bit data transfer). The data bus 
controller starts counting clock cycles when the M82786-CS and CYCLE-IN-PROGRESS signals 
are asserted. At the proper ti me (based upon clock counts), it asserts the read/write comm and. The 
data bus controller produces READY after receiving the SEN signal from the M82786. READY 
resets the count, and another read/write command is generated. 

The a ddress translator performs four fu nctions: i t converts the four byte enable signals to A , Aj , and 
BHE; i t incre ments A, after receiving READY for the first 16-bit transfer; it generates the clock 
signal (GBA L ) that latches the first 16-bit data word in the bidire ctional tr ansceivers when the 
80960MC processor performs a read operation; and it generates the READY signal for the CPU. 

Not shown is the cycle detector circuit that generates the C YGLE-IN-PROGRESS signal. This signal 
can be generated by using the circui t sim ilar to the one shown in Figure 5-2. The start of the cycle 
can be de tected by gating the ADS and DEN signals. The end of the cycle can be indicated by 
READY. 
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Figure 5-9: Operational Flow Diagram for M82786 Interface Circuit 



Operation 



The interaction between the M82786 and the 80960MC processor is summarized in Figure 5-9. The 
operation is divided into two 16-bit data movements for either a read or write operation. 
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The 80960MC processor generates a memory-mapped address and data for the desired graphics 
memory location. It accesses the M82786 by triggering the interfa ce c ircuit to gen erate the chip 
select signal and several operational signals: the read (RD) or write (WR) command, BHE, and the 
memory or I/O (M/IO) signal. The M82786 begins the memory operation after it completes the 
current graphics processing activity. The M82786 acknowledges that it is performing a memory 
operation by asserting SEN. 

After the M82786 asserts SEN, it begins a 16-bit memory read or write operation by translating the 
address inputs (A 21 through A ) to a multiplexed DRAM address, and generating the DRAM control 
signals. Note that A A and A are derived from the byte enable signals. 



Fo r a read o peration, the data bus controller uses SEN to generate the REA DY n si gnal. The assertion 
of READ Y causes the address translator to increment A, and to generate GB A L , which latches the 
lower 16 data bits on the B inputs of the bidirectional transceivers to the A side. 

Similarly, f or a write operation, the data bus controller uses SEN to generate the READY signal. The 
assertion of READ Y causes the address translator to increment A r The data buffer control uses SEN 
and the byte enable signals to produce GAB L , which enable the outputs for the lower 16 data bits of 
the bidirectional transceivers. 

The M82786 then deasserts SEN and the transfer of the first 16 data bits is complete. To transfer the 
seco n d 16 d ata bits, the interface circuit requests another memory operation by generating RD (or 
WR), BHE, and M/IO (CS is already asserted). After it completes the current graphics processing 
activity, the M82786 begins the memory operation and asserts SEN. 



For a read operation, the data b us con troll er uses SEN to generate the READY signal. The data 
buffer control uses SEN to assert GB A H and GB A L , which enable the outputs for the higher and lower 
16 data bits. 



For a write operation, the data bus controller uses SEN to generate the READY signal. The data 
buffer control uses SEN and the byte enable signals to produce GAB H , which enable the outputs for 
the higher 16 data bits of the bidirectional transceivers. 



The address translator generates READY for the 80960MC processor from the second READ Y to 
terminate the data transfer to the graphics memdry. 

SUMMARY 

The 80960MC processor supports 8-bit, 16-bit, and 32-bit I/O interfaces. A general system interface 
circuit can be designed that connects to many slave-type peripherals. This interface can be expanded 
to accommodate a bus master peripheral or a 32-bit to 16-bit data bus translator. These interfaces 
were illustrated by three design examples. 
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CHAPTER 6 
80960MC MULTIPROCESSOR SYSTEM ARCHITECTURE 

This chapter illustrates the flexibility of the 80960MC system architecture using the advanced 32- 
bit 80960MC processor and the 82965 Bus Extension Unit (BXU) in a multiprocessor design. 
System configurations are examined from a general perspective to highlight overall the design 
concepts. The details of system design are discussed in subsequent chapters. 

OVERVIEW OF A MULTIPROCESSOR SYSTEM ARCHITECTURE 

Modules form the natural boundaries for the hardware system architecture, as shown in Figure 6-1. 
Each module consists of components attached to its own local bus (L-bus). The modules are 
interconnected by a high bandwidth 32-bit multiplexed and packetized Advanced Processor (AP) 
bus, which transfers data at a peak rate of 42M bytes per second at 16 MHz. The 82965 Bus Extension 
Unit (BXU) provides the gateway between the L-bus and AP-bus, and performs several other 
functions, such as supporting an optional L-bus cache. 
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Figure 6-1 : Basic 80960MC System Configuration 



The L-bus 



The L-bus is used to connect the components within the module, which may include processors, 
memory arrays, and peripherals. In a multiprocessor system, each module contains an L-bus, which 
is typically confined to a single board. 

As described in the detail in chapter 3, the 32-bit L-bus is a high bandwidth, multiplexed bus which 
supports burst transactions, and can access up to four data words at a maximum rate of one word every 
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bus cycle. The L-bus consists of two groups of signals: address/data and control. The bus ha s a single 
fixed timing, although bus transactions can be lengthened through the use of the READY signal(to 
insert wait states). 

The L-bus protocol permits both primary and secondary bus masters to coexist on the bus. The 
secondary bus master must obtain use of the L-bus from the bus master through the use of the HOLDR 
and HLDAR signals. In a multiprocessor environment, a BXU is always used as a master in a 
memory module and is often used as a slave in a processor module. 

Complete details of the L-bus and bus operations are discussed in Chapter 3. 

The Advanced Processor Bus 

The AP-bus connects the 80960MC processor modules to system memory modules and I/O modules. 
The AP-bus is a synchronous, packetized, 32-bit wide bus. Synchronous refers to the fact that all 
components in the system, including 80960MC processors and BXUs must be driven by the same 
clock. It is considered a packetized bus, because read and write transactions are encoded in pairs of 
request and reply packets. 

The AP-bus is composed of four signal groups: 1) packet signals consisting of 32 multiplexed 
address/data lines and 6 packet specification lines, 2) transaction control signals, 3) error signals, and 
4) synchronization and initialization signals. 

A request packet flows from the requester to the server and a reply packet flows from the server to 
the requester. A request and reply packet may be separated in time to allow other transactions to use 
the bus in the intervening period. Thus, the AP-bus protocol supports pipelining of request packets 
to enhance bus utilization. Three transactions can be outstanding on the bus at any given time. 

Bus arbitration is decentralized and all bus agents monitor the bus activity. The bus supports fault 
tolerance by providing two bits of interlaced parity for the 38 packet signals, and two error report 
lines that are used to inform all system components when an error occurs. 

The topmost 16M bytes in the 32-bit address space of the bus are mapped (to coincide with the L- 
bus) for special bus transactions called Inter-Agent Communications (IACs). IACs are used for 
communication between 80960MC processors and for accessing the internal registers of the BXUs. 

The BXU is the only component that directly attaches to the AP-bus, and it contains all necessary 
bus interface logic. BXUs connect to each other in the form of a matrix to allow orderly growth in 
the system by the addition of AP buses or modules. An 80960MC system may have up to 32 logical 
modules (the practical limitation may be 20 modules because of electrical limitations) and four AP- 
buses. 

A more detailed discussion of the AP-bus is contained in Chapter 7. 
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The BXU Component 

The 80960MC processor and the 82965 BXU are the central components in the multiprocessor 
system architecture. The BXU interconnects the L-bus and the AP-bus arid implements the following 
functions: 

1 . Translation of L-bus requests from the 80960MC processor to request packets on the AP-bus. 

2. Support for cache on the L-bus 

3. A prefetch function for processors 

4. Support for the interface to a memory subsystem 

The BXU contains seven logic blocks: the AP-Bus Interface logic, the L-Bus Interface logic, Cache 
Support logic, Memory Support logic, IAC Support logic, I/O Prefetch logic, and Fault Tolerance 
logic. 

The AP-Bus Interface 

The AP-bus interface of the BXU provides the AP-bus signals. The AP-bus interface performs 
several AP-bus functions: arbitration, pipeline monitoring, address recognition, and AP-bus signal 
generation. The AP-bus interface provides a set of registers that allow software to define the address 
ranges and modes of operation. 

If the system design uses multiple AP-buses, then each module uses an individual BXU to interface 
to each of the AP-bus's. The AP-bus interface of each BXU provides address recognizers that can 
be set to a predefined range of addresses. The AP-bus address space can be interleaved over two or 
four BXUs to enhance system performance. Up to three accesses from the module may be pending 
on any pair of AP-buses. 

The L-Bus Interface 

The L-bus interface provides a direct interface between the BXU and the L-bus. The L-bus interface 
can be programmed to act as either a bus master or bus slave. The address recognizers of the L-bus 
interface support multiple memory address ranges. When there is more than one BXU on the L-bus, 
each L-bus interface coordinates the activity of the BXUs to provide efficient operation with multiple 
AP-buses. In this case, the address recognizers of the L-bus interface may be set up to support 
interleaving between multiple AP-buses. This ensures that AP-bus utilization is shared approxi- 
mately equally among the AP-buses in the system. 

Cache Directory and Control Logic 

The L-bus cache reduces the AP-bus traffic and increases overall system performance. The AP-bus 
traffic is reduced because the cache effectively diverts many system memory accesses to local 
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memory accesses. For example, when a read access is located in the cache memory, the BXU services 
that request directly from the cache, without generating corresponding bus cycles on the system bus. 
This boosts the system's performance because the cache can provide data to the requester on the 
L-bus much faster than requests that are serviced over a AP-bus by a memory module. 

The BXU provides the cache directory, the control logic, and the coherency logic. The coherency 
logic ensures that 80960MC processor uses the most recent data that is in global memory. 

The data storage resides on the L-bus in external SRAM components to allow access to a large cache. 
This memory is configured as a two-way set associative cache. 

Memory Support Logic 

The BXU provides specific support facilities for memory modules. The control of a memory module 
is done jointly between the BXU and a memory controller. The BXU provides the signals for the 
AP-bus interface and access to specific registers maintained by the memory controller. The memory 
controller is responsible for the detailed timing, control, and direct interfacing of the memory 
components. Normally the memory components are DRAMS, but other memory types can be 
supported as well. 

IAC Support Logic 

IAC messages are used by 80960MC processors to communicate interrupts and other information 
on the AP-bus. Because the 80960MC processors are not directly attached to the AP-bus, the BXUs 
act as the receivers for IAC messages on behalf of the 80960MC processors in their module. 

The BXU provides two registers for each of the two processors that can reside in a module. These 
registers store the data message and indicate the priority of the message. When an IAC message is 
received for one of the processors, the BXU checks the priority of the message and the status of the 
message data storage. Based on this information, the message is either accepted or rejected. 

I/O Prefetch Logic 

The I/O Prefetch logic of the BXU increases bandwidth and decreases latency for I/O accesses. Two 
prefetch channels handle the sequential I/O data transfers. Each channel contains two 16-byte 
buffers. After an I/O channel is initialized by receipt of a start command, the requests are serviced 
by the channel prefetch buffers. 

As data is requested from one of the buffers, the BXU automatically prefetches the next data block, 
and stores it in the other buffer. The prefetch function provides a significant increase in I/O 
performance because the data requests are handled immediately from the prefetch buffers. 
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Fault Tolerance Support 

In fault-tolerant systems with two or four AP-buses, the BXUs in a module operate in pairs as backup 
components for each other. For example, if an AP-bus fails, the BXU for that bus isolates itself from 
the failed bus, while the backup BXU services the requests directed to the failed module. This action 
of the BXU allows duplicate AP-buses to guard against single bus failures. To provide backup 
capabilities, each BXU tracks the accesses that are handled by its partner BXU. The BXU provides 
this fault tolerance logic, which is described in Part III of this manual. 

Modes of Operation 

The BXU has two modes of operation: PROCESSOR MODE or MEMORY MODE. When 
Processor mode is selected, the BXU can operate as a L-bus master or slave. In this mode, the BXU 
supports the cache, I/O prefetch, and IAC message functions. When Memory mode is selected, the 
BXU always operates as a bus master and generates the required L-bus signals. 

A more detailed discussion of the BXU is contained in Chapter 8. 

SYSTEM CONFIGURATIONS 

A multiprocessor 80960MC based system is composed of a set of modules connected to AP-buses. 
Figure 6-2 shows three types of modules: active, passive, and the combination of an active and 
passive (active/passive). These three types of modules are used to build a multiprocessor system. 
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MODULE 



£ ACTIVE /PASSIVE 
MODULE 
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Figure 6-2: Types of Modules 
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Active Module 

The active module consists of at least one 80960MC processor, an 82965 BXU, and the L-bus. Private 
or cache memory may be attached to the L-bus in this module. The BXU translates the L-bus signals 
to AP-bus signals. r 

In the active Modules, the L-bus traffic flows either on the L-bus to its resources (cache, etc) or flows 
through the BXU onto the AP-bus. No request packets, however, flow from the AP-bus through the 
BXU to the L-bus. 

For IAC transactions, the BXU acts as an extension of th e pro cessors. The BXU informs the 
processor of the impending IAC transaction by asserting the IAC pin of the 80960MC processor. 

In an active module, the BXU is set to the Processor Mode. When operating in this mode, the BXU 
does not need control of the bus, and consequently, does not need to arbitrate for control of the L- 
bus. 

Passive Module 

The passive module consists of memory or a slave I/O device, the BXU, and the L-bus. The L-bus 
connects the BXU to the memory or I/O device. The BXU translates AP-bus signals to L-bus signals. 

In the passive modules, all L-bus requests originate from a bus master (generally, the 80960MC 
processor) of an active module via the AP-bus. Bus traffic flows from the bus master through the 
requesting BXU attached to the active module, onto the AP-bus, and finally through the serving BXU 
attached to the passive module. For this case, the serving BXU (attached to the passive module) is 
set in the Memory Mode arid acts as a bus master. Slave I/O devices can be accessed by using 
memory-mapped addresses that are not within the IAC address space. 

Active/Passive Module 

Active/passive modules contain processors and memory, or master and slave I/O devices. This type 
of module can access a passive module, and has limited access to other Active/Passive modules. 

System Implications 

The active and active/passive modules in a system are connected to all system buses. Passive 
modules may be connected to a subset of the system buses. For example, if the system design uses 
four AP-buses; the active and active/passive modules will be connected to all four AP-buses; 
whereas, the passive mbdule may be connected to any or all of the AP-buses. Guidelines for the 
configurations are given below: 

p Each module can be connected to as many as four system AP-buses. Each system bus connection 
requires at least one BXU. 
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Each module can support up to two 80960MC processors (limited by the BXU ' s support for I AC 
messages). The number of other components is only limited by the electrical and physical 
constraints of the module implementation. 

• Logical addressing allows up to 32 modules for every system, although electrical considerations 
may limit the practical number of modules to 20. 

SUMMARY 

The basic hardware system configuration has been designed to be both modular and flexible. It is 
comprised of active, passive, and active/passive modules that form natural system boundaries. The 
high-bandwidth AP-bus is used for the datapath between the modules. Each module contains a BXU, 
which interfaces directly to the AP-bus and L-bus. To accomplish this task, the BXU contains seven 
logic blocks. 

This chapter presented an overview for basic hardware system design. The next three chapters 
discuss the details of the AP-bus, the BXU, and the memory and I/O interface. 
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CHAPTER 7 
ADVANCED PROCESSOR BUS 

Efficient bus utilization is essential in a multiprocessing system. A simple and efficient approach to 
building an 80960MC processor interconnect system is to use the Advanced Processor bus (AP-bus) 
The AP-bus protocol divides bus transactions into request and reply packets. Up to three request 
packets can be outstanding on the AP-bus before a reply packet must be received. In this way, the 
AP-bus reduces bus occupancy and increases the performance of 80960MC multiprocessor systems. 

This chapter describes the AP-bus operation and covers the topics shown below. 

• An overview of the AP-bus topology and description of the AP-bus signals 

• Memory organization and AP-bus transactions including memory and IAC transactions 

• AP-bus protocol and signal timing 

AP-BUS OVERVIEW 

The AP-bus forms the data communication path between the system modules. Access to the AP-bus 
is made possible by the BXU. The 80960MC processor utilizes the AP-bus to fetch instructions and 
to manipulate information from both memory and I/O devices. The AP-bus has the following 
features: 

• 32-bit multiplexed address/data path 

• High data bandwidth 

• Four- word burst capability 

• High bus utilization by supporting up to three outstanding requests at one time with intermixed 
requests and replies. 

• Transparent arbitration that allows the addition/removal of bus agents without modifying any 
hardware. 

• Interlaced parity 

The AP-bus is a synchronous packet bus that consists of several signal groups: packet signals that 
include 32 multiplexed address/data lines and six packet specification lines, transaction control 
signals, error signals, and synchronization and initialization signals. A transaction on the bus is 
separated into a request packet that flows from the requester to the server and a reply packet that flows 
from the server to the requester. 

A request and reply may be separated in time to allow other transactions to use the bus in the 
intervening period. This provides a pipeline feature that enhances bus utilization. Bus arbitration 
is decentralized and all bus agents monitor the bus activity. The AP-bus supports fault tolerance by 
providing interlaced parity over the address and packet specification lines, signal duplication on the 
transaction control lines, and a bus timer used to monitor the AP-bus for non-response to an 
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outstanding request. Redundant error report lines are provided to inform all system components 
when an error occurs. 

AP-Bus Topology 

Figure 7-1 shows a typical system configuration that uses two AP-buses (up to four AP-buses are 
allowed). The BXU provides the interface between the L-bus and the AP-bus. The BXU contains 
several programmable registers, which control its operation. See Chapter 8 for a detailed description 
of the BXU programmable register array. 
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Figure 7-1 : AP-Bus Topology 

Each BXU (bus agent) has three identification values: logical, physical, and bus. All the bus agents 
within the same logical module have the same logical identification value, which is assigned by the 
system software. This value is stored in the Logical Identification (Logical-ID) register of the BXU 
(see Appendix A for the description of the Logical-ID register). Each BXU on a particular bus has 
a unique physical identification assigned during initialization. Each bus has a unique identification 
value (zero through three). 

There are two types of agents: requesting and serving. The requesting agents are the BXUs attached 
to the active or active/passive modules. These BXUs translate the 80960MC processor signals to the 
AP-bus signals. The serving BXU receives the AP-bus signals, performs the desired functions, and 
passes the data back to the requesting 80960MC processor/ Bus agents do not initiate any AP-bus 
operations; they simply carry out the operations specified by the 80960MC processor. 
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AP-BUS SIGNAL GROUPS 

Figure 7-2 shows the four AP-bus signal groups: packet (address/data and specification lines), 
transaction control, error, and synchronization and initialization. This section presents general 
definitions of the AP-bus signal groups. Complete details of the signal descriptions and relationships 
are provided in subsequent sections. 



ADVANCED PROCESSOR BUS 






PACKET (38 LINES) K 

< > 

N TRANSACTION CONTROL (5 LINES) 

s — y 

^ ERROR (4 LINES) !f 

< _ => 



271079-108 



Figure 7-2: AP-Bus Signal Groups 



Packet Signal Group 



The Packet signal group consists of 38 bidirectional lines that transmit the address, data, and 
transaction type. The address and data signals are multiplexed. 



SPEC 5 -SPEC The Specification signals define the packet type and other parameters 

required for the bus transaction. Details of these signals are described in the 
"Bus Transaction" section of this chapter. These active low signals are open 
drain outputs of the BXU. 

AD 31 -AD The system Address/Data 31 through system Address/Data represent the 

address signals on the AP-bus during the address cycle and data signals on 
the AP-Bus during data cycles. AD 31 is the most significant bit and AD is 
the least significant bit. These active low signals are open drain outputs of 
the BXU. 
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Transaction Control Signal Group 

The Transaction control signal group consists of 5 bidirectional signa l s that control the sequencing 
of packets on the AP -bus. They are the four arbitration signals (ARB 3 -ARB ) and a signal that can 
defer a reply packet (RPYDEF). 



ARB 3 -ARB The four Arbitration signals determine which agent gains exclusive access 

to drive the AP-bus. Each of these signals must be common to all BXUs on 
a given AP-bus. The arbitration signals precede the address/data signals by 
one and one-half clock cycles. These active low signals are open drain 
outputs of the BXU. 



RPYDEF The Reply Deferral signal permits a serving BXU to defer sending a reply 

packet. Details for the use of this signal are in the "Reply Ordering" section 
of this chapter. This active low signal is an open drain output of the BXU. 

Error Signal Group 

The AP-bus incorporates two sets of signals to indicate an error has occurred. One set passes 
indications of errors to other BXUs, and the other set performs parity checking on the AP-bus packet 
signals. Although both are an integral part of a complete fault-tolerant support package, they may 
also be used in a non-fault-tolerant system implementation. 



CHK r CHK The Check signals provide interlaced even parity for the SPEC 5 -SPEC and 

AD 31 -AD lines. These active low signals are open drain outputs of the 
BXU. 



BERLj-BERL The Bus Error report line signals indicate that an error has been detected 

in the system. These active low signals are open drain outputs of the BXU. 

Synchronization and Initialization Signal Group 

The synchronization signal (CLK2) provides the capability for all AP-bus agents to operate with the 
proper timing relative to each other. The initialization signal (RESET) brings all AP-bus agents to 
a consistent state. 

CLK2 The System Clock provides the fundamental timing and synchronization for 

all transactions performed by the AP-bus agents. The clock rate is twice the 
frequency of a bus cycle. For example, if the bus cycle frequency is 16 MHz, 
then the CLK2 frequency is 32 MHz. 

RESET The assertion of the Reset signal forces all AP-bus agents to reset and 

synchronize. After the first system clock period begins, all bus agents 
remain in synchronization. The assertion of the RESET signal is the only 
method available to synchronize all the bus agents for proper operation. 
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Bus Signal Summary 

Table 7-1 shows the summary of the AP-bus signals. 

Table 7-1 : AP-Bus Signal Summary 



Signal Group 


Symbol 


Function 


Packet 




Specify the type of packet 


SPEC 5 -SPEC 


AD31-AD0 


32-bit address 


AD 31 -AD 


32-bit data 


Transaction 
Control 


ARB3-ARB0 


Arbitrate for AP-bus access 




Defers reply packet 


REPDEF 


Error 


CnKi-CrlKg 


Provide parity checks 




Indicate AP-bus error 


BERL r BERL 


Synchronization 
and Initialization 


CLK2 


Clock signal (double the bus frequency) 


RESET 


Resets bus agent to a known state 



NON AP-BUS MODULE SUPPORT SIGNALS 



The se sign als are point-to-point connections and are not considered part of the AP- bus. The INITID 
and COM signals are valid during initialization, and the MODCHK, BOUT, COM, and V REF signals 
are required for fault tolerant designs to detect the source of an error. POPQUE and SSBUS Y are used 
to co-ordinate activities between AP-Buses within a multiple bus system. 



INITID 



MODCHK 



BOUT 



The Initialization Identification signal pr ovides a way to access a single 
BXU during initialization. The INITID pin of the BX U is phy sically 
connected to one of the 32 AP-Bus address/data lines. The INITID signal 
together with "Identify Device Order" I AC provide a unique address for the 
bus agent during initialization. 

The Module Check signal is used in fault-tolerant system design and is 
described in detail in Chapter 1 1 . 

The Bus Output Control signal, when asserted, indicates that a bus agent 
is driving the AP-bus. This signal is used in fault-tolerant system design and 
is described in detail in Chapter 11. 
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COM 



The Communication signal is used to load information into a BXU as part 
of the initialization sequence or to inform external logic that the component 
has failed. This signal is not involved in any aspect of the AP-bus operation, 
but is provi ded to simplify loading board-dependent information into the 
BXU. The COM signal is also used to indicate external errors in fault- 
tolerant system designs. Complete details of this signal are provided in the 
"Serial COM Protocol" section of Chapter 14. 



The Voltage Reference pin provides a stable voltage reference for the AP- 
bus input buffers of the BXU. External hardware must provide a nominal 
2.0V on the VREF pin (see the M82965 BXU data sheet for the exact 
specification) during normal operation. The VREF pin is used to distinguish 
between a cold RESET (memory of errors cleared) and a warm RESET 
(memory of errors retained). Complete details about this signal are provided 
in Chapter 14. 



POPQUE 



The Pop Queue signal is used in fault tolerant system design and is 
described in Chapter 13. 



SSBUSY 



The Subsystem Busy signal is used to coordinate activity between AP- 
buses within a system. It is described in Chapter 13. 



MEMORY ORGANIZATION 

The AP-bus provides a four gigabyte address range to access memory and memory-mapped devices. 
The address range is accessible in four word blocks (each 32 bit word contains four 8- bit bytes). By 
organizing memory in this manner, the entire 4 Gbyte address range is logically accessible on word 
boundaries starting at address 0000 0000 H . The memory address (with the 4 least significant bits = 
0) present on the AP-Bus during the address cycle points to the first word within the block boundary. 
The source BXU can specify individual byte s withi n a word during a write transaction data cycle by 
encoding "BYTE MARKS" on the AP-Bus SPEC lines. 

A single AP-bus transaction can access a maximum of four words and cannot cross the boundary of 
a block. To achieve the lowest number of memory cycles for a transaction, single word accesses must 
be word aligned (least two significant bits = 0). A transaction may read or write a contiguous string 
of one to sixteen bytes of data within the block. The 80960MC processor will break any program's 
request of more than 16 bytes or accesses that would have necessitated crossing a block boundary 
into multiple transactions. 

The uppermost 16 Mbyte addresses of the 4 Gbyte address space are reserved for IACs; direct 
communication between bus agents (see the "IAC Transaction" section in this chapter). This specific 
set of memory-mapped addresses is recognized by all AP-bus agents to facilitate interagent message 
communications. IACs are used for system functions, such as initialization, access to registers in the 
AP-bus agents, and interrupt handling. 
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BUS TRANSACTIONS 

AP-bus agents communicate with each other by exchanging packets of information. A packet is a 
sequential group of bus cycles that contains information on the type of operation, the address 
location, and the number of data bytes to transfer. Two packets, a request and reply, form an AP- 
bus transaction. 

A request packet initiates an AP-bus transaction, and the reply packet completes the transaction. The 
request packet specifies three items: the type of operation, the location of the requested device, and 
the number of data bytes. The request packet includes data if a write operation is performed. The 
reply packet responds to the request packet by indicating the status of the action requested. For a read 
operation, the reply packet also responds with the requested data. 

Figure 7-3 shows the different types of request packets. The request packets are divided into two 
basic groups of operations: a read group that transfers data to the 80960MC processor, and a write 
group that transfers data from the 80960MC processor. Each group contains specific operations, 
such as read word(s) or Read Modify Write (RMW) operations. The read byte or read double-byte 
operations are optimized for memory-mapped I/O devices to facilitate the interface to an 8-bit or 16- 
bit bus. The RMW operations are used to perform "Atomic Accesses". In an Atomic Access the BXU 
guarantees that once a processor begins a read-modify-write operation on a set of memory locations, 
it is allowed to complete the operation before another processor is allowed to access the same 
location. The specification lines are used to indicate which function is performed. Complete details 
of the specific operations are described in this section. 







1 REQUEST PACKET 1 


Ml 1 1 


1 READ 


I WRITE I 




— READ 1 WORD 


— WRITE 1 WORD 




— — READ 2 WORDS 


— WRITE 2 WORDS 






— - READ 3 WORDS 


— WRITE 3 WORDS 






— - READ 4 WORDS 


— WRITE 4 WORDS 






— RMW-READ 1 WORD 


— - RMW-WRITE 1 WORD 






— RMW-READ 2 WORDS 


— RMW-WRITE 2 WORDS 






— RMW-READ 3 WORDS 


— RMW-WRITE 3 WORDS 






— - RMW-READ 4 WORDS 


— RMW-WRITE 4 WORDS 






— - READ BYTE 






— - READ DOUBLE BYTE 
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Figure 7-3: Request Packet Organization 
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The reply packets are also divided into two basic status groups that indicate acceptance or refusal of 
the requested operation, as listed in Figure 7-4. Each group contains specific responses, which are 
discussed in this section. 









REPLY PACKETS | 




i i 




ACCEPTED 


| REFUSED J 






— READ REPLY 1 WORD 

— READ REPLY 2 WORDS 

— READ REPLY 3 WORDS 

— READ REPLY 4 WORDS 

— WRITE-ACKNOWLEDGE (ACK) 


— REISSUE 

— NO-ACKNOWLEDGMENT (NACK) 

— BAD-ACCESS 
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Figure 7-4: Reply Packet Organization 



Transaction Specification 



To convey the typ e of operation, the request and reply packets use the SPEC 5 -SPEC lines. The six 
active low SPEC lines indicate the type of operation or status during the fir st bus cycle. On 
subsequent bus cycles, these lines are interpreted differently: for w rite op erati ons, SP EC A -SPEC t are 
used to indicate which data byte is valid; and for read operations, SPEC 4 and SPECj lines are set to 
a value of one to indicate a read transaction. 



The specification lines are divided into four data fields. These signals apply only during the first 
bus cycle. 



REQUEST/(SPEC.) 



This bit identifies the packet type. When it is low, this bit indicates the packet 
is a request packet. When it is high, it indicates the packet is a reply packet. 



MULTICYCLE (SPEC) 



When it is asserted, this bit indicates that the packet occurs in one bus cycle. 



CYCLE COUNT (SPEC 3 -SPEC 2 ) 

These bits along with the REQUEST and MULTICYCLE bits specify the 
length of the packet. 



OPERATION/STATUS (SPEC^SPEC^ 

These two bits identify the specific operation or status conveyed by the 
packet. 
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Table 7-2 shows the relationship between the encoded SPEC lines and th e speci fic operation of the 
request or reply packet during the first bus cycle. Note that the encoded SPEC lines do not use all 
the possible combinations. The values that are not encoded are unused and reserved for future 
product enhancements. Table 7-2 also shows the number of bus cycles and the number of words 
requested or transferred for a particular packet. 



Table 7-2: Specification Encodings for Packets 



Packet 


Category 


Function 


Specification Lines 




Bus 
Cycles 


Number of 

Words 

Requested 

(Transferred) 


(SPEC5-SPEC0) 


Binary 


Hexi- 
Decimal 
Equiv. 


5 4 3 2 10 


Request 


Read 


Read 1 word 


10 


20 




1 


Read 2 words 


10 10 


24 




2 


Read 3 words 


10 10 


28 




3 


Read 4 words 


1 1 10 


2C 




4 


RMW-Read 1 word 


10 1 


21 




1 


RMW-Read 2 words 


10 10 1 


25 




2 


RMW-Read 3 words 


10 10 1 


29 




3 


RMW-Read 4 words 


10 110 1 


2D 




4 


Read byte 


10 10 


22 




0.25 


Read double byte 


10 11 


23 




0.5 


Write 


Write 1 word 


110 


30 


2 


(1) 


Write 2 words 


110 10 


34 


3 


(2) 


Write 3 words 


1 11 


38 


4 


(3) 


Write 4 words 


11110 


3C 


5 


(4) 


RMW-Write 1 word 


110 1 


31 


2 


(1) 


RMW-Write 2 words 


110 10 1 


35 


3 


(2) 


RMW-Write 3 words 


111001 


39 


4 


(3) 


RMW-Write 4 words 


11110 1 


3D 


5 


(4) 
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Table 7-2: Specification Encodings for Packets (cont.) 



Packet 


Category 


Function 


Specification Lines 
(SPEC5-SPEC0) 


Bus 
Cycles 


Number of 

Words 

Requested 

(Transferred) 


Binary 


Hexi- 

Decimal 

Equiv. 


5 4 3 2 10 


Reply 


Accepted 


Read reply 1 word 


110 


06 


1 


d) 


Read reply 2 words 


10 10 


12 


2 


(2) 


Read reply 3 words 


10 110 


16 


3 


(3) 


Read reply 4 words 


110 10 


1A 


4 


(4) 


Write-Acknowledge 


10 


02 


1 


N/A 


Refused 


Reissue 


1111 


OF 


1 


N/A 


No-Acknowledgment 


1 1 


0A 


1 


N/A 


Bad-access 


10 11 


0B 


1 


N/A 



NOTE: Binary 1 indicates that the referenced SPEC line is asserted (i.e., driven low). 



After the first bus cycle, the SPEC lines represent other information for the write and read operations. 
For a write operation, four of the SPEC lines are used to specify which data bytes to write during the 
current bus cycle (see the "Write Data Transfer" section of this chapter for more details). For a read 
operation, three SPEC lines are used to indicate that data is being transferred (see the "Read Data 
Transfer" section for more details). 

Request Packets and Accepted Replies 

Data is transferred on the AP-bus by request and reply packets. A write-request packet transfers data 
from the requesting agent to the serving agent. A read-reply packet transfers data from the serving 
agent to the requesting agent. 

A word (32-bits) is the basic unit of measure for a data transfer, although individual data bytes (8 bits) 
can be transferred in a write request packet. Individual data bytes are transferred during a write 
operation by using four SPEC lines after the first bus cycle as byte marks. For a read operation, the 
entire data word is transferred and the 80960MC processor extracts the desired data bytes. 
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The write-request packets or the read-reply packets transfer up to four words of data on the AD lines 
in two to five bus cycles depending on the amount of data to transfer (a four- word write-request is 
five bus cycles long - one bus cycle to transmit the address and four bus cycles to transmit the data). 
For multiple word data accesses that do not start at word boundaries (i.e., Worc^-Bytej ), the BXU 
may require an additional cycle to properly align the data bytes before transferring them onto the AP- 
bus. The following examples of read and write packet transactions contained in this section illustrate 
details of these functions. 

Read Data Transfer 

The following sequence of events occurs during a read operation: 

• An 80960MC processor initiates a read operation on the L-bus by issuing a read command, 
supplying an address, and designating how many words to read through the SIZE signals. 

• The requesting bus agent (BXU) translates these signals into a read-request packet on the AP- 
bus. 

• The serving bus agent retrieves the data from the memory or I/O device and sends a read-reply 
packet to the requesting bus agent. 

• The requesting bus agent receives the data and transfers it to the L-bus, thus completing the 
cycle. 

The read transaction transfers one to four words of data. The address in the read-request packet points 
to the first data word to be read. Read transactions transfer a single word (Byte 3 , Byte 2 , Byte p and 
Byte ) in one bus cycle. Additional words require additional bus cycles. 

Individual bytes can be read, but only if the 80960MC processor extracts the desired data bytes and 
ignores the others. The BXU does not have the capability to fragment a word, and thus will always 
place a whole word on the L-bus. To read a string of data bytes that cross a word boundary within 
a block requires additional bus cycles, even if only two data bytes are requested. For example, if the 
desired data string is Word -Byte 3 and Wordj-Byte^ two bus cycles are required because two words 
are read: one to read Word and the other to read Word r In this case, the 80960MC receives both 
words and extracts the bytes desired. 

The following example depicts the data movement for a typical read transaction. Assume that an 
80960MC processor needs to read three words from a single block in memory located at 0000 0040 H . 
Table 7-3 shows the memory block organization with the letters A through P representing 16 
individual data bytes of the four words. This transaction is comprised of a one cycle request packet 
and a 3-cycle reply packet. 

The requesting bus agent places a request packet on the AP-bus, as shown in Table 7-4. The single- 
cycle request packet specifies that a three- word read operation is requested at location 0000 0040 H . 
The serving bus agent uses the read-request packet to read three words from the specified block of 
memory. The four byte word is the basic unit of transfer. 
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Table 7-3: Memory Block Data for Read Example 
Byte 3 Byte 2 Byte, Byte 

Word 3 

Word 2 

Word, 

Wordo 

NOTE: * Address 0000 0040 h points to this location. 
Letters represent one data byte. 

Table 7-4: Read-Request Packet 



p 





N 


M 


L 


K 


J 


1 


H 


G 


F 


E 


D 


C 


B 


A* 





Specification Lines 


Address/Data Lines 




5 


4 


3 


2 


1 





Byte 3 


Byte 2 


Byte, 


Byte 


Bus Cycle, 


1 





1 











oo H 


oo H 


oo H 


40 H 



The transaction is completed by a three-cycle read-reply packet that contains the requested 
information. Table 7-5 shows the alignment of the data bytes with respect to the AD lines in the reply 
packet. Because three words are read, the transaction requires three bus cycles: the first bus cycle 
contains the data word of the original address, the s econd a nd third bus cycles follow with the second 
and third data words. During the first bus cycle SPEC 5 -SPEC contain the binary code 10110 , 
indicating a "READ REPL Y 3 WORD PACKET". During t he second and third buscy cles, SPEC 5 
remains deasserted, SPEC 4 and SPECj remain asserted and SPEC 3 , SPEC 2 , and SPEC are "don't 
cares". 

Table 7-5: Read-Reply Packet 





Specification Lines 


Address/Data 




5 


4 


3 


2 


1 





Byte 3 


Byte 2 


Byte, 


Byte 


Bus Cycle, 





1 





1 


1 





D 


C 


B 


A 


Bus Cycle 2 





1 


X 


X 


1 


X 


H 


G 


F 


E 


Bus Cycle 3 





1 


X 


X 


1 


X 


L 


K 


J 


I 



NOTES: 

1 . Capital letters represent one data byte. 

2. "x" means "don't care." 
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Read Byte and Read Double-Byte 

The read byte or read double-byte are special request packets that can facilitate memory-mapped 
operations to peripheral devices on an 8-bit or 16-bit bus attached to a BXU on an I/O module. By 
specifying whether an access to a memory-mapped device is a byte or double-byte, the serving bus 
agent on the AP-bus can formulate a proper request its L-Bus. 

The read-byte request accesses a byte in a single cycle by using the two low-order AP-bus address 
bits to point to the requested data byte. The data byte is transferred in its normal location on the AD 
lines of the AP-bus. The read double-byte accesses any two-byte string within a block of address 
space. If the address points at the last byte of a word, then two bus cycles are required to transfer the 
two bytes as they overlap a word boundary. 

Both of these special packets are useful when the serving bus agent interfaces to either an 8-bit or 
16-bit local bus. For normal memory accesses (i.e., for non-memory-mapped accesses), these 
request packets operate like the read-request packet. 

Write Data Transfer 

The following sequence of events that occur during a write operation, are similar to a read operation, 
except that the data flows from the source agent. 

• An 80960MC processor initiates a write operation on its L-bus by issuing a write command, 
supplying an address, and designating how many words to write. 

• The requesting bus agent (BXU) translates these signals into a write-request packet on the AP- 
bus and sends the data to the serving bus agent. 

• The serving bus agent receives the data from the requesting bus agent and sends commands on 
its L-bus attached to write to the appropriate memory or I/O location. 

• The serving bus agent sends a reply packet thus completing the cycle. 

The write transaction transfers one to four words of data, with the capability to write to individual 
data bytes. The write-request packet transfers a single word in two bus cycles, a double word in three 
bus cycles, etc. The first cycle transmits the operation and address and, subsequent cycles transmit 
the data and the byte identification. To write a string of data bytes that cross a word boundary within 
a block requires additional bus cycles; one for each word boundary crossed. 

Because the write transaction modifies individual data bytes, the write-request packet must include 
information on wh ich by tes t o alter . After the first bus cycle, the packet contains this byte 
information on four SPEC lines (SPEC^-SPEC^), whi ch are byte mark signals during this tim e. Eac h 
byte mark corresponds to a d ata byt e: byte Mark (SPECj) represents B yte n , b yte Markj (SPEC 2 ) 
represents Byte p byte Mark 2 (SPEC 3 ) represents Byte 2 , and byte Mark 3 (SPEC 4 ) represents Byte 3 . 
The de sired data byte is written by asserting the appropriate byte mark. After the first bus cycle, 
SPEC 5 remains asserted to indicate that data is transferred for this write request. 
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The following example depicts the data movement for a typical write transaction. In this example, 
the 80960MC processor writes to six bytes starting at memory location 0000 004 1 H , which is located 
in memory block 0000 0040 H . Table 7-6 shows the memory block organization with the letters A 
through P representing 16 individual data bytes. This transaction comprises of a three cycle write 
packet and a one cycle reply (ack or nack) packet. 

Table 7-6: Memory Block Data Before Write Operation 
Byte 3 Byte 2 Byte, Byte 

Word 3 

Word 2 

Word, 

Wordo 

NOTE: Letters represent one data byte. 

The three-cycle write-request packet is generated by the requesting bus agent. Table 7-7 illustrates 
the "WRITE-2-WORDS" command, the desired address, and the alignment of data byte marks and 
data on the AP-bus. The first cycle defines the specific operation and transmits the word aligned 
address. During the second bus cycle, the BXU transfers the data bytes of Worc^ ( U, V, and W), 
which are designated by the byte marks. During the third bus cycle, the designated data bytes of 
Word, (X, Y, and Z) are transferred. 

Table 7-7: Write-Request Packet 



p 
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K 
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B 


A 





Specification Lines 


Address/Data 
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1 





Byte 3 
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Byte, 


Byte 


Bus Cycle, 


1 


1 
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BusCycle 2 
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1 
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1 
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V 


u 


. — 


BusCycle 3 


1 





1 


1 


1 


x 


— . 


z 


Y 


X 



NOTES: 

1 . Capital letters represent one data byte. 

2. "x" means "don't care." 

3. "— " means no change to current value. 



The serving agent sends a write acknowledge reply packet that signifies completion of the transaction 
as shown in Table 7-8. The updated memory block organization is shown in Table 7-9. 
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Table 7-8: Write-Acknowledge Packet 
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Address/Data 
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V 


U 
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Table 7-9: Memory Block Data After Write 
Byte 3 Byte 2 Byte, Byte 

Word 3 
Word 2 
Word, 
Wordo 
NOTE: Letters represent one data byte. 

Read-Modify-Write 

The Read-Modify-Write (RMW) functions allow the AP-bus agents to read and modify data at a 
given location as a single indivisible action. The AP-bus protocol defines a RMW-READ packet to 
indicate the start of an indivisible operation and a RMW- WRITE packet to denote the termination 
of this action. 

In the memory mode, the BXU provides two to four RMW locks with timeouts. Four locks are 
available if the module is not interleaved with other modules, and two locks if it is interleaved. The 
serving BXU in a memory module locks one quarter of its available memory (up to 1 Gbyte), 
whenever a RMW-READ is accepted into the module and AP address bits 6 and 4 match its address 
segment. Any other RMW-READ packets addressing this segment are rejected. This Block of 
memory is available, however, for any other type of memory operation by any bus. For example, the 
source agent may make additional read or write memory accesses, but not RMW accesses. 

The RMW-READ packet can be answered with one of two reply packets: the READ-REPLY or 
REISSUE-REPLY. The read-reply packet returns the requested data and indicates that the lock for 
the appropriate Block was set. The reissue-reply packet indicates that the lock is already set and that 
the requesting agent must attempt the operation later. No data is returned with a reissue-reply packet. 

The RMW- WRITE packet writes data and removes the lock. This packet is equivalent to a normal 
write-request packet except that it resets the lock for the selected memory location. The write- 
acknowledge reply packet indicates that the lock has been removed. 
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Four independent lock timers are provided in the BXU to ensure a malfunctioning agent cannot 
initiate a lock and deny other agents access to the system memory, The timer for a particular segment 
starts when its lock is asserted, and should never timeout in normal system operations. If however, 
a lock timeout occurs, its associated lock will be removed. The duration of the timeout is between 
4096 and 8 192 clock cycles. 

When using the RMW operations, certain conventions must be observed by the bus agents. 

1. The purpose of a RMW operation is to construct indivisible transactions. These operations 
should not be used to directly lock a memory data structure. The lock must be restricted to a short 
duration operation. In general, this means that an agent performing an indivisible sequence must 
guarantee that it does not suspend operation or handle an interrupt without first terminating the 
indivisible sequence. 

2. All agents sharing a data structure or communication signal must use the same location as the 
starting point for their RMW operators. If the locations are not the same, the conflicting 
operations are not correctly sequenced (blocked). 

3. Any single agent may have only a single RMW operation outstanding at any given time. 

Refused Reply Packets 

Besides the various reply packets that accept a request, there are three reply packets that refuse a 
request packet. They are the reissue, no-acknowledgement, and bad-access replies. Each of these 
reply packets elicit different responses from the requesting bus agent. 

The REISSUE reply packet, which was briefly described in the "Read-Modify-Write" section, 
indicates that the serving agent was unable to reply at the prescribed time because of temporary 
conditions. There is no problem that prevents a successful access if the request packet is sent at a 
later time. The reissue reply packet is transmitted in response to a RMW-READ packet when 
memory is already locked, or for a request packet directed to a memory-mapped I/O device where 
the attached I/O bus is temporarily blocked. 

The NO- ACKNOWLEDGEMENT reply packet (NACK) indicates that the request packet cannot 
be completed. The requesting agent must determine how to respond to this condition. NACK reply 
packets are used exclusively for IAC messages. For example, the NACK reply packet is issued in 
response to an IAC request if the serving agent has insufficient buffer space, or if the request packet 
does not have high enough priority. Complete details are discussed in the following "IAC 
transaction" section. 

The BAD-ACCESS reply causes the requesting agent to terminate a transaction. One possible cause 
of a BAD- ACCESS simply is the transaction exceeded the bus timeout period. The requesting agent 
terminates its own transaction with a bad-access reply to remove the request packet from the AP-bus. 
The bad-access reply can be used for a variety of situations to indicate problems with the request 
packet. When the bad-access reply is encountered, the requesting processor should not try the access 
again. 
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The BAD- ACCESS reply signifies a failed access and is a valid reply to any request packet. The 
accessed address should be considered unavailable. The source agent must determine the appropriate 
recovery from this condition. 

IAC TRANSACTIONS 

An Interagent Communication (IAC) is a mechanism to facilitate communication between the 
80960MC processors. All bus agents recognize a specific set of memory-mapped addresses (the 
uppermost 16M of the 4G-byte address range) as an IAC transaction. 

IAC requests are split into two major groups: message and register requests. Message IACs provide 
a method for one processor to communicate with another processor. The register-request IACs allow 
direct communication to the registers in the BXU. 

The information represented within a specific IAC address is: an FF H in the uppermost eight bits (high 
order byte) are an IAC identification (function); the remaining 24 bits define the type of IAC, the 
Module Destination, L-Bus Destination, and the Internal Destination in the BXU. 

IAC Flow 

The 80960MC processor originates the IAC transaction by writing to a reserved memory-mapped 
address (top 16 Mbyte) on its L-bus. The BXU that is attached to the 80960MC processor's L-bus 
recognizes that address as an IAC request, and responds by performing the requested action itself or 
by passing the IAC request to another BXU via the message passing AP-bus. IACs always originate 
on an L-Bus. If an IAC message is transmitted on the AP-bus, the IAC request flows from an L-bus, 
onto the AP-bus, and then to another L-bus, which is its final destination. 

The 80960MC processor can be connected to a maximum of four AP-buses through four BXUs, but 
only the AP-bus designated as the message bus during initialization transmits the IAC message. The 
BXUs that coexist on both the L-bus and the message AP-bus handle the IAC transactions generated 
by the 80960MC processors. 

IAC Address Formats 

The IAC is defined by a 32-bit memory-mapped address, which contains the following five address 
fields: IAC identification, module destination, L-bus destination, IAC type, and internal destination, 
as shown in Figure 7-5 . The IAC identification field provides a way to designate the memory -mapped 
address as an IAC transaction. When this field is equal to FF H , an IAC transaction occurs. The module 
destination field specifies which specific BXU residing on the AP-bus responds to the IAC request. 
The L-bus destination field specifically identifies which of four possible BXU's residing on the 
processors local bus will either act on the request or propagate it onto the AP-Bus. The IAC type field 
specifies the type of IAC transaction. The multipurpose internal destination field specifies various 
commands, BXU register addresses, or IAC message priorities. 
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Figure 7-5: IAC Address Format 

The 80960MC processor can initiate the six types of IAC transactions listed below. In addition, the 
80960MC processor can reserve address space on the L-bus for its own use. This section provides 
the details for each of the following IAC transactions. 

IAC Message (IAC Type 001 1 B ) 

Register request using a logical address (IAC Type 0010 B ) 

Register request using a physical address (IAC Type 0100 B ) 

Register request to BXUs on the L-bus (IAC Type 0000 B ) 

Identify device order (IAC Type 01 1 1 B ) 

Private L-bus space for the 80960MC processor (IAC Type 1 1 1 1 B ) 

AC access type 1 1 1 1 B reserves address space on the L-bus for the 80960MC processors exclusive 
use. All addresses of this access type are ignored by the BXU. This allows the 80960MC processor 
to perform interrupt acknowledge handshakes, etc. 



IAC Message (IAC Type 001 1 B ) 

This IAC transaction passes messages to and from the 80960MC processors. Figure 7-6 shows the 
IAC message address format. 

The IAC identification field is equal to FF H to indicate that this is an IAC transaction. 

The module destination field is separated into two categories: the unit-identification bits (six bits 
labeled UUUUUU), and a processor-identifier bit (labeled N). The low order bit in the module 
destination field is not used and can be any value (indicated by the shaded area). 

To specify the module destination of a message, a logical identification number is assigned to the six 
high-order bits of the module destination field. The logical identification number, located in the 
Logical-ID register of the BXU, is assigned by software and must uniquely identify a logical module. 
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Figure 7-6: Address Format for IAC Message Transaction 

Because two processors can reside on the L-bus, the processor-identifier bit(N) provides a way to 
select which of the two processors connected to the destination BXU will receive the IAC message. 
The serving BXU uses the Processor-Priority register associated with destination processor to 
determine which actions to execute in response to the IAC message. 

The L-bus destination field is not used. 

The IAC type field is set to 001 1 B to indicate that this is an "IAC message". 

The internal destination field is used to communicate the priority of the IAC message. The five bits 
labeled PPPPP are used by the BXU to determine whether the IAC is accepted or rejected. There are 
thirty-one levels of priority; PPPPP=0 is the lowest priority and PPPPP=31 is the highest priority. 
The four low-order bits of this field are set to zero to align the address on a 1 6-by te boundary because 
the IAC message can be up to 16 bytes long. The high order bit in this field is not used and can be 
any value (indicated by the shaded area). 

Register Request Using a Logical Address (IAC Type 001 B ) 

This IAC transaction allows the 80960MC processor to access a register in a BXU using the logical 
address of a module. This type of IAC transaction is used primarily in fault-tolerant systems where 
multiple components operate as a single logical unit (see Part III for complete details). Figure 7-7 
shows the address format for this IAC transaction. 

The IAC identification field is equal to FF H to indicate that this is an IAC transaction. 

The module destination field is separated into two categories: the unit-identification bits (six bits 
labeled UUUUUU), and an access bit (labeled A). The low order bit in the module destination field 
is not used and can be any value (indicated by the shaded area). 
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Figure 7-7: Address Format to Access a Register Using a Logical Address 

To specify the module destination, a logical identification number is assigned to the six high-order 
bits of the module destination field. The logical identification number, located in the Logical-ID 
register of the BXU, is assigned by software and must uniquely identify a logical module. 

The purpose of the access bit is to allow testing the fault tolerant circuitry in a single BXU, even 
though that BXU may be part of a module that consists of two or four identical BXU' s with the same 
logical address. When the access bit is asserted, the Access-Register will be used to determine which 
of the individual agents of the single logical module will respond to the IAC. This bit is used for the 
testing and start up of fault-tolerant system designs, and is described in detail in Chapter 12. 

The L-bus destination field (labeled BB) indicates which BXU on the L-bus passes the IAC request 
to the AP-bus for multiple bus configurations. The BXUs are identified by the ID of the AP-bus to 
which they are attached. The combination of the L-bus destination field and the module destination 
field uniquely identifies one bus agent as the destination of the IAC request. 

The IAC type field is set to 0010 B to specify that this IAC transaction accesses a register in a BXU 
using a logical address. 

The internal destination field specifies a particular command or register of the BXU. The two low- 
order bits on the AP-bus are equal to zero because all register and command addresses are word 
aligned. The data written into a register is typically one word long, and can access only one 
register/command per transaction. The data word sent with a command is ignored by the destination 
BXU. The IAC request must match the register/command size for a valid request. If the request is 
invalid, the serving BXU will respond with a NACK reply packet. The BXU ignores the byte enable 
signals on all IAC requests. 

Register Request using a Physical Address (IAC Type 01 00 B ) 

This IAC transaction allows the 80960MC processor to access a register in a BXU using a physical 
address. This type of IAC transaction permits access to bus agents before the final logical 
configuration is established. Figure 7-8 shows the address format for this IAC transaction. 

The IAC identification field is equal to FF H to indicate that this is an IAC transaction. 
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Figure 7-8: Address Format to Access a Register Using a Physical Address 

The Module Destination field is separated into two categories: a class identification (labeled K), and 
a component identification (labeled CCCCC). The class ID is set during manufacturing, and is used 
to distinguish a BXU from other future AP-Bus components. The class ID for a BXU is zero. 

The BXU compares the value of the module destination field to the value in its Physical-ID register 
(see Appendix A for the description of the Physical-ID register). If a match occurs, the BXU responds 
to the IAC memory-mapped address. The two low order bits in the module destination are not used 
and can be any value (indicated by the shaded area). 

The L-bus destination field (labeled BB) indicates which BXU on the L-bus passes the IAC request 
to the AP-bus for multiple bus configurations. The BXUs are identified by the ID of the AP-bus to 
which they are attached. The combination of the L-bus destination field and the module destination 
field uniquely identifies one bus agent as the destination of the IAC request. 

The IAC type field is set to 0100 B to specify that this IAC transaction accesses a register in the BXU 
by using a physical address. 

The internal destination field specifies a particular command or register of the BXU. The two low- 
order bits on the AP-bus are equal to zero because all register and command addresses are word 
aligned. The data written into a register is typically one word long, and can access only one register/ 
command per transaction. The data word sent with a command is ignored by the destination BXU. 
The IAC request must match the register/command size for a valid request. If the request is invalid, 
the serving BXU will respond with a NACK reply packet. The BXU ignores the byte enable signals 
on all IAC requests. 

Register Request On the L-Bus (IAC Type 0000 B ) 

This IAC transaction is provided for initialization purposes. It gives the 80960MC processor access 
to the message-control registers of the BXU on the L-bus. Figure 7-9 shows the address format for 
this IAC transaction. 

The IAC identification field is equal to FF H to indicate that this is an IAC transaction. 
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Figure 7-9: Address Format to Access a Register From the L-Bus 

The module andL-bus destination fields are not used because the IAC transaction remains on the L- 
bus and does not propagate onto the AP-bus. A write operation using this IAC will write data to all 
BXU's residing on the L-Bus. A read operation is performed only by the message BXU in a multiple 
AP-bus configuration, unless the access is to the IAC message buffer. In this case, the 80960 will 
read data from the BXU whose IAC Message-Data-Valid bit in its corresponding Processor-Priority 
register is set. Normally, this is the message BXU, but may be another BXU under certain error 
conditions. 

The IAC type field is set to 0000 B to specify that this IAC transaction accesses a register of a BXU 
on the processors local bus. 

The internal destination field specifies a particular command or register of the BXU. The two low- 
order bits on the AP-bus are equal to zero because all register and command addresses are word 
aligned. The data written into a register is typically one word long, and can access only one register/ 
command per transaction. The data word sent with a command is ignored by the destination BXU. 
The IAC request mus t match th e register/c ommand size for a valid request. If the request is invalid 
the BXU will return a BAD AC instead of a READY on the L-bus. The BXU ignores the byte enable 
signals on all IAC requests. 

Identify Device Order (IAC Type 0111 B ) 

Identify Device Order is a special IAC that supports initialization of the system. This IAC is used 
to assign logical and physical identification values to a particular BXU. Complete details are 
provided in Chapter 14. Figure 7-10 shows the address format for this type of IAC transaction. 

The IAC identification field is equal to FF H to indicate that this is an IAC transaction. 

The module destination field is not used for this type of t ransacti on. To determine the component 
destination, this IAC uses the Initialization Identification (INITID) pin on the BXU, which is tied to 
one of the AP-bus address/data lines. This IAC request is transmitted to all the BXUs on a specific 
AP bus. The individual BXU addressed by this IAC is determined by asserting one (and only one) 
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of the t hirty-tw o bits in the first data word. If the asserted AD line is physically connected to the 
BXU ' s INITID pin, that BXU will accept the I AC. All other BXU ' s will ignore this I AC transaction. 
(See Chapter 14 for complete details.) 
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Figure 7-10: Address Format to Identify a Device 

The addressed BXU does not send a reply packet to this type of IAC. Consequently, a time-out error 
is always generated by the requesting BXU. The time-out error causes the requesting BXU to 
ter minate the transaction by generating a BAD- ACCESS reply packet on the AP-bus and asserting 
the BAD AC pin on L-bus. The bad-access reply is used to clear the AP-bus pipeline. 

The L-bus destination field (labeled BB) indicates which BXU on the L-bus passes the IAC request 
to the AP-bus for multiple bus configurations. The BXUs are identified by the ID of the AP-bus to 
which they are attached. 

The IAC type field is set to 01 1 1 B to specify that this IAC transaction is an identify device order IAC. 

The internal destination field is not used. 

To guarantee correct operation, the addressed BXU must either be idle or processing an Identify 
Device Order IAC from its L-bus. Incorrect operation results if the addressed BXU is processing a 
memory reference or another type of IAC. This restriction is required because of the special address 
recognition and register loading that occurs as a result of this IAC. 

Summary of IAC Transactions 

The 80960 architecture's interagent communications is implemented with two general types of 
IAC's, register request IAC's and message passing IAC's. The register request IAC's allow a CPU 
to access any BXU in the system by its logical or physical address. The message passing IAC's 
facilitate communication between multiple CPU's and bus agents in multiprocessing and fault 
tolerant system implementations. A summary of the five data fields contained in each IAC is shown 
in Table 7-10. 
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Table 7-10: Summary of M82965 IAC Transactions 



Fields 

Addresses 
IAC Type 


IAC 
Ident. 
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LB 
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IAC 
Type 


Internal 
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3 2 
1 4 


2 1 

3 6 




1 1 
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O O 
9 O 


Register Request lAC's: 
CPU -» BXU Reg Req'st on L-Bus 
CPU -» BXU Reg Req'st (Logical Addr) 
CPU -► BXU Reg Req'st (Physical Addr) 

Message lAC's: 
IAC Message 
Identify Device Order 
CPU Private L-Bus Space 


11111111 
11111111 
11111111 

11111111 
11111111 
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UUUUUUA* 

occccc** 

UUUUUUN* 
«fftttt«t 
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0000 
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LEGEND: 
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Definition 


A 


Access bit 


B 


L-Bus destination (0-3) 


C 


Component ident no. (0-31) 


N 


Processor identifier bit 


P 


IAC priority (0-31) 


R 


BXU command or register 


U 


Unit identification no. 


* 


Don't care (0 or 1) 



AP-BUS PROTOCOL 

The Advances Processor Bus (AP-Bus) protocol provides a method to process transactions in an 
efficient and orderly manner. Figure 7-11 presents an overview on how the AP-bus agents (BXUs) 
handle transactions packets on the AP-bus. When the bus agent receives signals from an 80960MC 
processor that require use of the AP-bus, the bus agent arbit ra tes fo r acces s to th e AP- bus by using 
the four bi-directional, open-drain arbitration signals (ARB 3 , ARB 2 , ARB p and ARB ). The arbi- 
tration occurs during a group of bus cycles called a time-slice period. 

As a result of the arbitration, the BXUs enter a grant queue. The grant queue can hold up to four 
bus requests from the bus agents. Arbitration is suspended if the grant queue contains four requests. 
It resumes when there are fewer than four requests in the queue. This queue operates on a First-In 
First-Out(FIFO) basis. Thus, the first request to enter the grant queue is the first to exit the grant 
queue. Once the agent exits the grant queue, it gains access to the AP-bus to perform a bus transaction. 

By separating bus transactions into a request packet and a reply packet, transactions can be pipelined 
to maximize the bus bandwidth. After the bus agent puts a request packet on the bus and enters the 
transaction into the AP-bus pipeline, it waits for the reply packet. While an agent waits for a reply, 
other agents can gain access to the bus. The AP-bus protocol permits a maximum of three outstanding 
requests (transactions pending) in the AP-bus pipeline. Transactions are removed from the pipeline 
when the corresponding reply packet is received by the requesting agent. A new request packet can 
enter the pipeline when a slot becomes available. 
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Figure 7-11: AP-Bus Protocol 

The reply ordering process controls the order in which the reply packets are sent over the AP-bus, 
The BXU's bus sequencing control determines when the request packet is placed on the AP-bus and 
when to send a reply packet. The AP-bus protocol assures that no agent is locked from access to the 
bus. 

Arbitration 

The bus agen ts arbit rate among themselves to obtain access to the AP-bus. The arbitration process 
uses the ARB 3 - ARB signals and the Arbitration-ID register of the BXUs during a time-slice period 
to place the bus agents in order in the grant queue (see Appendix A for the description of the 
Arbitration-ID register). The following sections describe the details of the arbitration process and 
illustrates the process with an example. 

Arbitration Process 



Arbitration for the AP-Bus is accomplished by using the ARB a -ARB signals and the Arbitration- 
ID register of the BX U dur ing the time-slice period. The ARB signal is common to all BXUs on 
the same AP-bus, the ARBj signal is common to all BXUs on the same AP-bus, and so forth. All 
the BXUs on the AP-bus continuously monitor the arbitration signals. 



The time-slice period consists of a v ariabl e number of clock cycles. The time-slice period begins 
when the BXU's sense that all of the ARB signals are deasserted. The time-slice period is divided 
into a maximum of 16 time-step intervals to sequence the requesting agents into the grant queue. 
When all the requesting agents are placed in the grant queue, the time-slice period ends. 
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The Arbitration-ID register of the BXU specifies which time-steps the agent asserts its ARB lines 
during a time-slice period. The grant queue order is established by having the bus agents arbitrate 
at different time-step intervals during the time-slice period. 

The Arbitration-ID register, which must be set to a unique value during the initialization process, 
contains two fields: a Drive field and Count field. The Count field determines the time-step interval 
in which the BXU asserts the arbitration line specified by the Drive field. For example, with a count 
value of 0000 B , the BXU asserts the arbitration line specified by the Drive field during the first time- 
step interval. For a count value of 1 1 1 1 B , the BXU asserts the specified arbitration line during the 
sixteenth time-step interval. A maximum of three agents can arbitrate in a single time-step interval, 
but the agents are placed in the grant queue individually (the arbitration winner goes into the queue). 



The Drive field determines which of the three low order arbitration lines(ARB 2 -ARB ) are asserted 
during the time-step interval specifie d by the Count field. The Drive field indicates the a gent's 
priori ty in that time-step interval: the ARB line has the highest priority, followed by A RB,, th en 
ARB 2 . For example, consider three agents with the same co unt va lue, the agent asserting ARB is 
place d in the grant queue first, followe d by t he agent asserting ARB 1? and the n by th e agent asserting 
ARB 2 . In this case, one agent asserted ARBj for two cycles, another asserted ARB 2 for three cycles. 
The agent that asserted ARB 2 was placed in the grant queue during the last cycle of the time-step 
interval. 

All agents that a re not arbitrati ng du ring the first time-step interval, but need access to the AP-Bus, 
will assert ARB 3 . Assertio n of A RB ^ indicates that additional time-step intervals are required in the 
current time-slice period. ARB 3 will remain asserted as long as any bus agent has not gained access 
to the g rant q ueue. For example, assume that the Arbitration-ID registe r spec ifies that the bus agent 
asserts ARB during the fifteenth time-step interval. This agent asserts ARB 3 during the first time- 
step interval and keeps it asserted until the fifteenth time-step interval. 

All agents requesting the AP-bus must be ready to arbitrate in the first cycle of a time-slice period 
to be included in that time-slice period. Any agent that requires the AP-bus after the beginning of 
the time-slice period must wait until the next time-slice period. The duration of the time-slice period 
depends upon several factors: the number of agents requesting the AP-bus, the depth of the grant 
queue, and the count specification in the Arbitration-ID register. 

During a time-slice period, all of the BXUs are placed in proper order in the grant queue. A maximum 
of three bus agents can enter the grant queue during any given time-step interval. The grant queue 
contains four entries. When the grant queue is full, arbitration is automatically suspended and the 
arbitration lines are held constant. In this case, the time-step interval can be stretched beyond the 
normal maximum of three cycles by a full grant queue. Arbitration resumes when an agent's request 
enters a pipeline slot, creating an open slot in the grant queue. 

Arbitration Example 

The following arbitration example illustrates a typical sequence of events during the arbitration 
process. The bus agents vie for control of the AP-bus by arbitrating during the time-slice period. As 
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a result of the arbitration, the agents ' bus requests are assigned to the grant queue in the order specified 
by their respective Arbitration-ID registers. The agents enter the AP-Bus pipeline from the grant 
queue on a first-in first-out (FIFO) basis. When all the agents have been are assigned to the grant 
queue, the time-slice period ends. 

The following example shows how seven different bus agents (labeled A, B, C, D, E, F, and G) 
arbitrate for the AP-bus. The example assumes that the grant queue is empty and that transactions 
from three agents (X, Y, Z) are in AP-bus pipeline at the beginning of the time-slice period. 

Table 7-11 shows the value for the Arbitration-ID register of the seven BXU's used in this example. 

Table 7-1 1 : Value of the Arbitration-ID Register for Agents in the Example 



Agents 


Arbitration-ID 
Register 


Hexi-Decimal 

Equivalent 

for Drive 

Field' 


Hexi-Decimal 

Equivalent 

for Count 

Field 


Drive 


Count 


D 


D 


C 


c 


c 


c 


A 


























B 





1 














1 





C 


1 

















2 





D 

















1 





1 


E 





1 











1 


1 


1 


F 





1 








1 


1 


1 


3 


G 


1 











1 


1 


2 


3 



NOTE: * Assumes that the two high-order bits are zero. 



Figure 7-12 illustrates how the interaction of ARB 3 through ARB define the time-slice period and 
time-step intervals. 

During clock cycle 1 , agents A, B, C, D, E, F,and G are ready to arbitrate. Time-slice period N begins 
on the next clock cycle, because the four ARB lines are deasserted. At this time, the grant queue 
is empty, and the pipeline, which contains three slots, is full. 
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Figure 7-12: Arbitration Example 
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Time-slice Period N, Time-step Interval 

The following actions occur during time-slice period N, time-step interval 0. 

• Agents A, B, and C arbitrate during this time-step interval because the Count field in their 
respective Arbitration-ID registers has a value of zero. 

• Agent A is assigned to the grant queue first, followed by agent B, then agent C, because of the 
settin gs in the Drive field of the Arbitration-ID register (agent A asserts ARB , agent B asserts 
ARB r and agent C asserts ARB 2 ). Note that these ARB lines remain active until the agent is 
assigned to the grant queue. 



• Agents D, E, F, and G assert the ARB 3 line because they arbitrate in subsequent time-step 
intervals. Each agent holds its ARB 3 line low until it arbitrates in its specified time-step interval. 

Time-step interval ends at the end of bus cycle 4 because all of the agents arbitrating in this time- 
step interval are in the grant queue. 

Time-slice Period N, Time-step Interval 1 

The following actions occur during time-slice period N, time-step interval 1. 

• Agents D and E arbitrate during this time-step interval because their Count field in the 
Arbitration-ID register has a value of one. Thus, at the start of time-step interval 1 agents D and 
E deassert ARB 3 , and assert one of their low order ARB lines. 

• Agent D is assigned to the grant queue first, followed by agent E, because of the settings in the 
Drive field of their respective Arbitration-ID registers (agent D asserts ARB , and agent E asserts 
ARBj). Note that the request from Agent E cannot be immediately placed into the grant queue, 
because the queue is temporarily full Consequently, Agent E is assigned to the grant queue after 
Agent A obtains a slot in the AP-Bus pipeline (a reply packet for agent Z's outstanding request 
was shown at the end of bus cycle 6 to complete its transaction; thus, opening a slot in the pipeline 
for Agent A). 



Agents F and G continue to assert their ARB 3 lines because they arbitrate in subsequent time- 
step intervals. Each agent holds its ARB 3 line low until it is assigned to the grant queue. 

The example shows Agents B and C requesting an additional access to the AP-bus during bus 
cycle 6. Because time-slice period N was already in progress when these agents made their 
second request, they must wait until a new time-slice period begins. Remember, a bus agent must 
be ready to arbitrate in the first cycle of a time-slice period to be included in that period. These 
agents are listed in the row labeled Agents Pending Arbitration in Figure 7-12. 

The pending transaction for agent Y is completed at the end of bus cycle 8 to open a slot in the 
AP-Bus pipeline. 
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Time-step interval 1 ends at the end of bus cycle 8 because all of the agents arbitrating in this time- 
step interval are in the grant queue. 

Time-slice Period N, Time-step Interval 2 

The following actions occur during time-slice peribd N, time-step interval 2. 

• No agents arbitrate during this time-step interval because none of the Arbitration-ID registers 
of the requesting agents specify arbitrate 

• The example shows Agent D requesting an additional access to the AP-bus during bus cycle 9. 
Because the time-slice period N was already in progress, this agent must wait until a new time- 
sliqe period begins. Agent D joins agents B and C in the Agents Pending Arbitration row 
shown in Figure 7-12. 



• Agents F and G continue to assert their ARB 3 lines because they arbitrate in subsequent time- 
step intervals. Each agent asserts its ARB 3 line until it arbitrates in its specified time-step 
interval. 

Time-step interval 2 ends at the end of bus cycle 9. 

Time-slice Period N, Time-step Interval 3 

The following actions occur during time-slice period N, time-step interval 3. 

• The request from Agent B exits the grant queue and takes the next available slot in the pipeline. 

• Agents F and G arbitrate during this time-step interval because the Count field in their 
Arbitration-ID registers have a value of three. Thus, at the start of time- step interval 3 agents F 
and G deassert ARB 3 , and assert one of their low order ARB lines. Because all of the agents 
requesting AP-Bus access at the beginning of this time-slice period are either in the AP-Bus 
pipeline, the grant queue, or in this time-step interval; the common ARB 3 line is now deasserted. 

• Agent F is assigned to the grant queue first, followed by agent G, because of the settings in their 
respective Drive fields (agent F asserts ARB , and agent G asserts ARB 2 ). Note that the request 
from Agent G cannot be immediately placed into the grant queue because the queue is 
temporarily full. Consequently, Agent G is assigned to the grant queue after Agent C obtains 
a pipeline slot, opening a position in the grant queue (the transaction for agent X is completed 
at the end of the bus cycle 10, opening a slot in the pipeline for Agent C). 



Agent G deasserts ARB 2 at the end of bus cycle 12. At this point all of the ARB lines are de asserted. 
This marks the end of time-step interval 3 and time-slice period N. Since all of the ARB lines are 
deasserted in bus cycle 13, the agents pending arbitration (Agents B, C, arid D) will initiate a new 
time-slice period to arbitrate for access to the AP-bus. 
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Reply Ordering 

The AP-bus protocol supports pipelining of up to three request packets on the AP-bus. An AP-Bus 
Transaction is made up of a request packet and matching reply packet. The request packets enter a 
pipeline with three available slots. When the requesting bus agent receives a reply packet in response 
to its request packet, the transaction is completed. This completion of the request-reply transaction 
opens an slot in the pipeline for another request packet to enter. Reply ordering is the process that 
determines the order of reply packets on the AP-bus. 

The slots in the AP-Bus pipeline are designated slot 1, slot 2, and slot 3. Slot 1 is the beginning of 
the pipeline and contains the oldest request packet. Slot 2 contains the request packet that has been 
there the second longest time, and slot 3 contains the most recent request packet. 

The reply packets are normally returned in the same order that the request packets were placed in the 
pipeline. For example, consider the case where the request packet for slot 1 is sent on the AP-bus. 
When the bus requesting agent receives the reply packet, the transaction is completed, and then 
removed from slot 1 . At this time the request packets in slot 2 and 3 increment one position to allow 
for a new request packet in slot 3. The next reply packet is in response to the request packet that 
moved from slot 2 to slot 1 . 

The ordering of reply packets may be modified by the serving Bus Agent. If the serving BXU needs 
an extended length of time to send a reply pa cket (to inh ibit a bus timeout), i t may defer its place in 
the pipeline by asserting the Reply Deferral (RPYDEF) signal. If RPYDEF is asser ted, the req uest 
packet currently associated with slot 1 is placed at the end of the pipeline in slot 3. RPYDEF may 
only be asserted by the serving agent that needs to send a reply packet in response to the request packet 
in slot 1 . 

Bus Sequencing 

The reply ordering process defines how reply packets are placed in position on the AP-bus. Bus 
sequencing defines how request and reply packets are intermixed on the AP-bus. 

To determine th e bus act i vity of the ne xt cycle (H + 1 clock cycle), the bus sequencing process samples 
the ARB 3 -ARB , SPEC -SPEC 5 , and RPYDEF signals in the previous cycle (N-l clock cycle), and 
checks the state of the grant queue and pipeline of the current cycle (N cycle), as shown in Figure 

7-13. 

The following rules govern bus sequencing: 

1 . If an agent is at the top of the grant queue and the pipeline is not full (less than three transactions), 
then the agent's request packet is directed to the AP-bus in the next available cycle. The next 
available cycle refers to the bus cycle following the request packet presently being transmitted 
on the AP-bus, or the current cycle if there is no request packet on the bus. 

2. If the pipeline is full (three transactions in progress), then a reply packet will take the next 
available bus cycle. 

3. If the grant queue is empty and the pipeline is not full, then a reply packet will take the next 
available bus cycle. 
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Figure 7-13: Bus Sequencing 

The first rule keeps the pipeline as full as possible by giving precedence to request packets over reply 
packets. In this way, the BXUs maximize the use of the available AP-bus bandwidth. 

After the pipeline is full, a reply packet will be placed on the AP-bus, as stated by the second rule. 
A reply packet completes a transaction to open up a slot in the pipeline. This activity puts the first 
rule into effect again. 

AP-BUS SIGNAL TIMING 

The AP-bus signals are referenced to a common system clock (CLK2), which operates at twice the 
frequency of a bus cycle. A bus cycle defines the period of time for information to be transferred on 
the AP-bus. 

Figure 7-14 shows the relationships between CLK2 and the AP-bus signals. The AP-bus uses three- 
quarters of the available bus cycle for address and data transmission. The address and data lines are 
driven on edge D and sampled on edge C. The remaining one-quarter of the bus cycle is allocated 
for clock skew and signal hold time. 

Because the AP-bus is a synchronous bus, the clock skew must be no greater than 7 ns over the entire 
bus. To achieve this the traces on the system backplane (AP-bus) must be of equal length between 
the clock/reset drivers and each socket (BXU) on the AP-bus. This will ensure all BXU's will drive 
and sample the AP-bus synchroniously. Also its extremely important that the clock and ARB lines 
be noise free. Noise on the these lines can cause the system to exhibit system arbitration anomolies 
not easily identified during debug. 

It is critical that all processors and BXU's within the system are sychronized to the same CLK2 
A edge. The Clock Phase Sychronization and RESET Timing discussed in Chapter 14 and the 
RESET setup and Hold times specified on the 80960M C and 82965 data sheets must be strictly 
adhered to for the BXU's to properly arbitrate for the AP-bus. 
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The ARB 3 -ARB , CHK f CHK , and BERL f BERL signals use the same basic AP-Bus timing 
concept except that they are offset by one CLK2 bus cycle. 



AP-BUS 

ADDRESS/DATA 

CYCLE 
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Figure 7-14: AP-Bus Signal Timing 



The AR B and CHK signals lead the AD, SPEC , and RPYDEF signals by one CLK2 cycle, and the 
BERL lines lag by one CLK2 cycle. The CHK signals (bus parity signals) are shifted into the next 
bus cycle to allow time for generation of parity from the internal data that has been transmitted. The 
CHKj-CHKq lines are sampled one clock period after the data is sampled and compared against the 
internally calculated parity of the received data. The BERL lines are similarly shifted into the 
following bus cycle to allow the BXU's internal circuit ry time to verify the received data, in 
anticipation of an Error Reporting cycle. (Note, the BERL lines operate at one-half the speed of the 
AP-bus, i.e. CLK-M,) 



General AP-Bus Timing 

Most input signals (called group one) on the AP-bus are sampled on the risin g edge of CLK2atth e 
beginn ing of phase 2 (edg e C), as shown in Figure 7-15. The exceptions are the CHK^CHK^ BERL r 
BERL , and ARB 3 -ARB signals (called group two), which are sampled on the rising edge of CLK2 
at the beginning of phase 1 (edge A), as shown in Figure 7- 1 6. Regardless of the clock edge, the setup 
and hold times are the same. 

The output signals for group one are driven relative to the falling edge of CLK2 at the middle of phase 
2 (edge D), the output signals for group two are driven on the falling edge of CLK2 at the middle of 
phase 1 (edge B). See Figures 7-15 and 7-1-6. 
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Figure 7-15: AP-Bus Timing for Group One Signals 
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Figure 7-16: AP-Bus Timing for Group Two Signals 
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AP-Bus Timing Considerations 

When designing a system using the AP-bus, the system propagation delays should be considered. 
This propagation time must allow for settling of ringing, ground shift, and crosstalk; all of which are 
dependent on board and system materials and design. The following equation can be used to 
determine the propagation delay, given a specific clock frequency. 

T = 2T - [T. + (T . or T . ) + T, + T .] 

prop eye L h v cd dz 7 ds skew J 

where: 

T ro = Propagation delay 
2T c c = Two system clock periods 
T h = System clock high time 

T cd = Clock to data valid delay time (see 82965 data sheet T17) 

T dz = Clock to high impedance delay time 

T ds = Data setup time 

T skew = System clock skew 
The system clock skew is defined as follows: 

T . = T h + T . - T .. where: 

skew h oh dh 

T oh =Output hold time 
T dh =Input data hold time 

SUMMARY 

The AP-bus is a synchronous, packetized 32-bit wide bus used to communicate between Agents 
(BXU's) on behalf of L-Bus residents. Transactions on the AP-bus are executed by using request and 
reply packets. Special memory-mapped transactions (IAC's) allow the 80960MC processor and 
BXUs to communicate with each other. The AP-bus protocol defines the arbitration, bus sequencing, 
and reply ordering. Arbitration is performed by the bus agents as they continuously monitor the 
arbitration signals, to ensure that all bus agents have access to the bus. 
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CHAPTER 8 
AP-BUS INTERFACE USING THE BXU 

The M82965 Bus Extension Unit (BXU) is the key component in building multiprocessor designs 
with the 80960MC processor family. The BXUs connect to each other in an expandable matrix that 
can support up to 20 processor and memory modules in a single, high performance system. The BXU 
increases overall system performance by providing hardware support for local caches, I/O prefetch, 
message passing, and multiprocessor arbitration. 

This chapter describes the operation of the BXU and covers the topics described below. 

• An overview of the major logical units of the BXU, which includes a description of the modes 
of operation and a summary of the registers 

• Details of each major logical units of the BXU 

• Diagnostic support functions and performance of the BXU 

• System configurations 

The BXU is also the key to building a fault-tolerant systems with the 80960MC processor family. 
Through redundant modules, fault-tolerant systems based on the BXU can sustain a single-point 
failure, and then automatically reconfigure themselves, while application programs continue without 
disruption. Simply by replicating components, a 80960MC system can be built that can sustain a 
failure of any single processor, memory array, or bus, and then reconfigure itself. The error detection, 
isolation, and recovery functions are built in the hardware. 

Part III of this manual describes the fault-tolerant design as well as the fault-tolerant aspects of the 
BXU. This chapter focuses on the multiprocessor design aspects of the BXU. 

BXU FUNCTIONAL OVERVIEW 

The primary function of the BXU is to connect the Local Bus (L-bus) with the system-wide Advanced 
Processor bus (AP-bus). In this way, the BXU allows the system to expand incrementally as each 
new module or system bus is added. Other functions of the BXU include the following: 

Isolation for the AP-bus to allow the construction of large, expandable, module 

• Low latency, high-bandwidth memory accesses for sequential data streams used in I/O transfer 
loops 

• Support for multiprocessor systems by reducing a processor ' s effective demand for system bus 
bandwidth, allowing the connection of modules to multiple system buses (thereby increasing the 
total bandwidth available to the module), and providing support registers for communication 
between processors 

Support for a cache memory on the L-bus. 
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During normal operation, the BXU is transparent to accesses directed from one bus to the other. For 
example, if a BXU recognizes an address that lies in its assigned memory address space, it either 
generates an identical access on the other attached bus or services the request from the cache. 

Major Logical Units 

The BXU is composed of seven major logical units: 

AP-bus Interface 

L-Bus Interface 

Cache Directory and Control Logic 

Memory Support Logic 

IAC Support Logic 

I/O Prefetch Logic 

Fault Tolerance Support 

Individual sections describe the details of operation for each logic unit. The fault tolerance logic unit 
is described in Part III. 

Modes of Operations 

The BXU operates in either Processor or Memory mode, and the interpretation of several signals 
changes depending on the mode of the BXU. Processor mode is intended to support modules that 
are composed of a 80960MC processor or combinations of 80960MC processors, memory, and I/O 
devices. Memory mode p rovides support for memory arrays. The mode of operation is loaded during 
RESET from the LERL 2 pin and stored in the LBI Control Register. 

Processor Mode 

Processor mode provides efficient service to modules composed primarily of 80960MC processors. 
The BXU supports the cache, prefetch, and I AC message functions. In apure processor module, there 
is no need for the BXU to participate in L-bus arbitration because it operates as a slave to the CPU. 

In Processor mode, the BXU can also handle modules with a processor and global memory (or master 
and slave I/O controllers). For this configuration, the BXU can act as both a master and a slave on 
the L-bus. Although a request flows in either direction through the BXU, the BXU is biased toward 
handling outbound requests. If there is a significant amount of traffic in both directions, performance 
can be degraded. 

Memory Mode 

In the Memory Mode, the BXU acts as an L-bus master in a memory module. No requests are 
accepted from the L-bus. All requests flow from the AP-bus into the module. When the BXU is in 
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the Memory Mode, it provides support for memory functions and signal generation. It does not 
provide the cache interface, prefetch, or IAC message handling functions. Memory mode is oriented 
toward supporting SRAMs, DRAMs, and the memory controllers. 

Table 8- 1 summarizes the status and configuration of the major functional blocks and the BXU pins 
for each mode. 

Table 8-1 : Summary of BXU Modes of Operation 







Mode 


Notes 


ll«?lll 


Processor 


Memory 


Logic Unit 


AP-Bus Interface 


3 Inbound 
3 Outbound 


3 Inbound 
Outbound 




L-Bus Interface 


Master or 
Slave 


Master 




Cache Directory 
and Control Logic 


Available 


Not Available 




Support 
Function 


Processor Support 


Available 


Not Available 


1 


Memory Support 


Off 


On 




RMW Lock Support 


Off 


On 


2 


Dual 

Function 

Pins 


Mode Dependent 
Multiplexed Signals 


CR 


DT/R 


3 


CW 


DEN 


3 


lACo 


ERR 


3 


lAd 


FRF 


3 


WY 


COR 


3 


WY, 


MEM/REG 


3 


WD 


UNC 


3 


WD, 


ECC 


3 



NOTES: 

1 . The processor support handles the processing of IAC messages received on behalf of the processors 
in the module. 

2. The RMW lock support covers a wide range of possibilities. In Processor mode, the LOCK signal is used 
to indicate outbound RMW-Read and RMW-Write operations. In Memory mode, internal RMW locks are 
kept to support inbound RMW requests. 

3. The pin name for multiplexed signals represents the function of the pin for each particular mode. The 
pins not listed perform the same function in both modes. See the M82965 data sheet for a complete pin 
definition. 
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Register and Command Summary 

To provide a flexible interface to various system configurations, the BXU has numerous program- 
mable registers and commands. Complete details of the registers and commands are described in 
Appendix A. 

Initialization and control of the BXU is done by reading and writing to the BXU's registers with 
register IAC's. Table 8-2 lists all of the registers and commands in the BXU, and table 8-3 provides 
a memory-map of all the BXU's registers. For completeness, the list includes the registers and 
commands used in a fault-tolerant design, which are described in Part III. 

AP-BUS INTERFACE 

The AP-bus interface of the BXU requires no external logic circuits. The AP-bus interface logic 
performs several functions: address recognition, arbitration, pipeline monitoring, and the AP-bus 
signal generation. The AP-bus interface logic section of the BXU provides registers that allow 
software to define its address range. When the BXU recognizes its address from the AP-bus, it 
responds by either translating the request packet to control signals for the L-bus, or by handling the 
request internally. For additional request packets, the BXU processes the request packets in the order 
it receives them. To complete the transaction, the BXU generates the reply packets associated with 
the received request packets from the AP-bus. 

To maximize the use of the pipeline queue, the AP-bus interface provides six buffers, each capable 
of holding an entire bus transaction. Three buffers are allocated for outbound (to the AP-bus) 
requests and three for inbound (from the AP-bus) requests. 

Memory Address Recognition 

The address recognizer logic contained in the AP-bus interface partitions the address space into an 
L-bus address space and a system bus address space. The BXU does not alter the address that it passes 
to the L-bus. Except for the upper 16-Mbyte address locations reserved for IAC transactions, the 
address recognizer is used to match the address on request packets located in 4-Gbyte address range. 

The address recognizer consists of two registers: the AP -Match (068 H ) and AP-Mask (06C H ) registers 
(see Appendix A on for the description of these registers). The AP -Match register defines where the 
L-bus address space begins in the total address space on the system bus, and the AP-Mask register 
defines how much of the available address space on the system bus is mapped to the L-bus. Together, 
these two registers define a window that maps memory from the AP-bus to the L-bus. 

The AP-Mask register is used to mask the address bits that select a location in the L-bus address space. 
The size of the address space mapped to the L-bus determines the number of low-order zeros in the 
AP-Mask register. For example, to recognize 2 N bytes for transferral to the L-bus, the N low-order 
bits of this register are set to a value of zero. The upper bits from N to 3 1 are set to a value of one. 
The size of the mapping window ranges from 128-Kbytes to the entire 4-Gbytes. 
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Table 8-2: Summary of BXU Registers and Commands 



AP-Bus Interface (2) 


Register 


Address 


PHYSICAL-ID 


040 H 


LOGICAL-ID 


044 H 


COMPONENT-SPECIFIER (1) 


048 H 


ARBITRATION-ID (1) 


048 h 


COM 


04C h 


AP-CONTROL 


050 H 


FT1 


054 H 


MAXTIME 


058 H 


FRC-SPLITTING-CONTROL 


05C H 


TEST-DETECTION 


060 H 


FRC 


064 H 


AP-MATCH 


068 H 


AP-MASK 


06C H 



NOTES: 

(1) The component specifier (read only) and the 
Arbitration-ID (write-only) are physically two 
separate registers that share a common address. 

(2) The AP-Bus interface registers are the only 
registers that can be addressed using access 
mode. 



L-Bus Interface 


Register 


Address 


PHYSICAL-ID (LOCAL) 


140 H 


LOGICAL-ID (LOCAL) 


144 H 


LBI-CONTROL 


148 H 


SYSTEM-BUS-ID 


14C H 


CACHE-CONFIGURA TION 


150 H 


CACHE-TEST 


154 H 


LOCAL-BUS-TEST 


158 H 


MATCH 


160 h 


MASK 


164 h 


MATCH, 


168 H 


MASK, 


16C H 


MATCH 2 


170 h 


MASK 2 


174 H 


PRIVATE MEMORY MATCH 


178 H 


PRIVATE MEMORY MASK 


17C H 


Command 


Address 


INVALIDATE-CACHE 


15C H 




Memory 


Register 


Address 


LOCK 


300 H 


Externally Mapped 

Registers (Reserved 

for Memory Controller) 


340 H -37C H 
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Table 8-2: Summary of BXU Registers and Commands (cont.) 



IAC Message Support 


Register 


Address 


PROCESSOR-PRIORITY 


000 H 


PROCESSOR-MESSAGE 


010 H -01C H 


PROCESSOR-PRIORITY, 


020 h 


PROCESSOR-MESSAGE, 


030 h -03C h 




Prefetch Buffer Registers 


Channel 


Buffer 


Word 


Address 











3C0 H 








1 


3C4 h 








2 


3C8 h 








3 


3CC h 





1 





3D0 H 





1 


1 


3D4 H 





1 


2 


3D8 H 





1 


3 


3DC H 










3EO H 







1 


3E4 h 







2 


3E8 h 







3 


3EC h 




1 





3F0 h 




1 


1 


3F4 h 




1 


2 


3F8 h 




1 


3 


3FC h 



Prefetch 


Register 


Address 


PREFETCH-CONTROL 


240 H 



Fault Tolerance 


Register 


Address 


TEST-TYPE 


0C0 H 


SPOUSE-ID 


0C4 h 


QMR 


oeo h 


MODULE-ERROR-ID 


0E4 H 


BUS-ERROR-ID 


0E8 H 


ERROR-LOG 


oec h 


ERROR-RECORD 


ofo h 


FT2 


0F4 H 


Command 


Address 


TEST-REPORT 


104 H 


PRIMARY-CA TASTROPHE 


108 H 


SHADOW-CA TASTROPHE 


10C h 


TERMINA TE-PERMANENT 
ERROR-WINDOW 


110„ 


ATTACH-BUS 


114 H 


DETACH-BUS 


118„ 


SYNC-REFRESH 


11C„ 
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Table 8-3: M80960/M82965 Register Map 







Register Contents 




Register Function 


Internal 
Dest Addr (H) 
















(10 Bit Field) 


3 2 


2 1 


1 O 


O 






1 4 


3 6 


5 8 


7 O 


Processor Priority 


000 




PPPP 




PWVF 


* 


004 










* 


008 










* 


OOC 










Processor Message 


010 










Processor Message 


014 










Processor Message 


018 










Processor Message 


01 C 










Processor Priority 1 


020 




PPPP 




PWVF 


* 


024 










* 


028 










* 


02C 










Processor Message 1 


030 










Processor Message 1 


034 










Processor Message 1 


038 










Processor Message 1 


03C 










Physical-ID 


040 




KCCCCC 






Logical-ID 


044 




UUUUUU 






Read = Component-Specifier 


048 Read 






TTT 


TTTSSSSS 


Write = Arbitration ID 


048 Write 








DDCCCC 


Com 


04C 


cccccccc 


CCCCCCCC 


CCCCCCCC 


CCCCCCCC 


AP-Control 


050 








WADRRS 


FT1 


054 








WACI EB 


Maxtime 


058 








WTMMMM 


FRC-Splitting Control 


05C 








WMWPSF 


Test Detection 


060 








WTFFRR 


FRC 


064 








WRMCTM 


AP-Match 


068 


BBBBBBBB 


BBBBBB 




I I E 


AP-Mask 


06C 


MMMMMMMM 


M M M M M M 




NN 


* 


070 










* 


0BC 










Test Type 


OCO 




TTTT 


TTTTTTTT 


TTTTTTTT 


Spouse ID 


0C4 






SSSSSSSS 




* 
* 


0C8 










* 


0DC 










QMR 


0E0 








PSWSTWME 


Module Error ID 


0E4 






CCCCCCCC 


SSSSSSSP 


Bus Error ID 


0E8 






OO0OO0BB 


SSSSSSSP 


Error Log 


0EC 


EC 


CVTTTTTP 


CCCCCCCC 


SSSSSSSP 


Error Record 


0F0 




VTTTTTP 


CCCCCCCC 


SSSSSSSP 


FT2 


0F4 








S WFCWBR 


* 


0F8 










* 

Test-Report Command 


100 
104 












Primary-Catastrophe Command 


108 






Shadow-Catastrophe Command 


10C 






Term-Perm Err Wind Command 


110 


Data Word Ignored 




Attach-Bus Command 


114 






Detach-Bus Command 


118 






Synch-Refresh Command 


11C 
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Table 8-3: M80960/M82965 Register Map (cont.) 









Register Contents 




Register Function 


Internal 
DestAddr(H) 


















(10 Bit Field) 


3 2 


2 1 


1 


O 






1 4 


3 6 


5 8 


7 


* 


120 










* 


13C 










Physical-ID (Local) 


140 




KCCCCC 






Logical-ID (Local) 


144 




uuuuuu 






LBI-Control 


148 








E0DBI I NN 


System-Bus-ID (Read-only) 


14C 






ss 




Cache-Configuration 


150 








E T T Y I L 


Cache-Test 


154 








WHMHM 


Local-Bus-Test 


158 








MP I ARNTC 


Invalidate-Cache Command 


15C 


C 


ata Word Ignore 


j 




Match 


160 


BBBBBBBB 


BBBBBB 




CSFF 


MaskO 


164 


MMMMMMMM 


MMM M M M 






Match 1 


168 


BBBBBBBB 


BBBBBB 




CSFF 


Maskl 


16C 


MMMMMMMM 


MMM M M M 






Match 2 


170 


BBBBBBBB 


BBBBBB 




CSFF 


Mask 2 


174 


MMMMMMMM 


MM M M M M 






Private Memory Match 


178 


BBBBBBBB 


BBBBBB 




£ 


Private Memory Mask 


17C 


MMMMMMMM 


M M M M M M 






* 


180 










* 


23C 










Prefetch-Control 


240 








EAA 


* 


244 
2FC 










Lock 


300 








FTWLLLL 


* 


304 










* 


33C 












& 


10 


Externally Mapped 














Registers Reserved 














Memory Controller 


37 


r C 










* 


380 


* 


3BC 
3C0 










Channel 



Buffer 



Word 







3C4 








1 






3C8 








2 






3CC 








3 






3D0 





1 









3D4 





1 


1 






3D8 





1 


2 




Prefetch Buffer 


3DC 





1 


3 




Registers 


3E0 
3E4 









1 






3E8 







2 






3EC 







3 






3F0 




1 









3F4 




1 


1 






3F8 




1 


2 






3FC 




1 


3 





Indicates address not used 
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The AP -Match register defines the L-bus address space within the AP-bus address space. The N low- 
order bits of the AP -Match register are ignored, since they are masked by the AP-Mask register. This 
means that the L-bus address space must be aligned on integer multiples of the recognized range. 

The AP-Mask register should always be set up before the AP -Match register to avoid matching on 
all memory and IAC requests that are placed on the AP-bus. This occurs because 0000 H is the default 
value of the AP-Mask register. 

Figure 8-1 shows the basic hardware function provided by the address recognizer. Both the AP- 
Match register and the AP-bus address are masked by the AP-Mask register. The results are 
compared to determine if the address is recognized. 



31 


MATCH REGISTER 


18 31 


MASK REGISTER 


18 31 


AP-BUS ADDRESS 18 


I I I I I I I 


II I! II II 


I II II MM I II II 


I I I I I I I 




^ 


' 














'» 


\ 








,♦ 


r 


AND | 


| AND 


: 


















I EQUALITY TEST | 












* 

RECOGNIZE 




271079-123 



Figure 8-1 : Address Recognizer Functional Diagram 

Figure 8-2 illustrates how the AP -Match and AP-Mask registers are used to recognize an address. In 
general, the Recognize fields of the AP-Mask and AP -Match registers determine the location of the 
mapping windows. Assume that the AP-bus address is AAAA AAAA H , the AP-Mask register is set 
to value of FF00 0000 H , and the AP-Match register is set to a value of AA00 0001 H . The least 
significant bit in the AP-Match register enables the address recognizer assuming that the FT 1 and FT2 
registers (see Appendix A for the description of the FT1 register and the FT2 register) are properly 
set. The matching logic between the AP-Match and AP-Mask registers produces a result of 
10101010000000 B (or AA00 H assuming bit 17 and bit 16 have a value of zero). Similarly, the 
matching logic between the AP-Mask register and AP-bus address produces a result of 
10101010000000 B (or AA00 H assuming bit 17 and bit 16 have a value of zero). Thus, the equality 
test is true and an address match is accomplished. 

Guidelines for Recognizer Programming 

The programming guidelines listed below should be followed to guarantee consistent, predictable 
operation of the address recognition mechanism. The third guideline applies to a L-bus address 
recognizer, which is described in the "L-Bus Interface" section. 

1 . The AP-Mask register must always contain contiguous values of one in the upper bit positions 
and contiguous values of zero in the lower bit positions. 
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2. The L-bus address recognizers and the AP-bus address recognizers cannot overlap. 

3. For multiple BXUs on the same AP-bus, the AP-bus address recognizers cannot overlap. 

4. The address recognizers cannot overlap to the IAC address range. 



31 


AMTCHREGISTER 

RECOGNIZE FIELD is 31 


MASK REGISTER 
RECOGNIZE FIELD 


18 31 


AP-BUS ADDRESS 
RECOGNIZE FIELD 18 


hl°l 


i|o|i | | 1 


ommmmmuh 


1|1|1|1|1 


1|0|0|0|( 


>|0|0||l|0|l 


lolHoli 


ommmmm 




AAXX H 

1 






AAYY 




Lt 


FFOO H 




it H 



1 T 



AAOOn { { AAOOh 

I EQUALITY TEST I 

} 

RECOGNIZE 

NOTE: SHADED AREA INDICATES DON'T CARE CONDITION DUE TO THESE BITS BEING MASKED BY 
CORRESPONDING BITS-0 IN MASK REGISTER 



271 079-1 24B 



Figure 8-2: Address Recognizer Example 

Module (Address) Interleaving 

The AP-bus interface supports interleaving of request packets for inbound memory references. This 
is referred to as MODULE INTERLEAVING between multiple memory modules (on the same AP- 
bus). Interleaving consists of propagating requests on noncontiguous 16-byte boundaries with 16- 
byte granularity. The interleaving may be one-way (no interleaving), two-way or four- way . For one- 
way interleaving, the BXU processes every request that is recognized by its AP-bus memory address 
recognizer. When the system is two-way interleaved, the system bus address space is effectively 
divided into two partitions, with addresses 0000 H -000F H , 0020 H -002F H , ... assigned to the first 
partition and addresses 001 H -001F H , 0030 H -003F H , ... assigned to the second partition. A BXU will 
respond only when an access is recognized by the L-bus address recognizer and is within its assigned 
partition. Four- way interleaving functions like two-way, but four partitions exist, with the first one 
consisting of address 0000 H -000F H , 0040 H -004F H , etc. 

The BXU does not modify the address when it is passed to the L-bus. The interleaving functions are 
controlled by the Interleave Control fields in the AP -Match mdAP-Mask registers. These Interleave 
control fields are separate and independent from the interleaving control for the recognizers in the 
L-bus interface, which is described in the next section. 

When programming the AP -Match and AP-Mask registers, the address recognizers in the LBI- 
Control register of the BXU' s for bus interleaving must be set to cover the same address range. The 
total address space covered by a mapping window is dependent on the size of the recognized space 
and the number of ways interleaved. For example, consider a two-way interleaved system with 
address recognizers set for 256-Kbyte windows. In this case, each BXU actually recognizes only 
128-Kbytes, every other 16-byte block in a 256-Kbyte window. 
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L-BUS INTERFACE 

The L-bus interface provides a direct interface between the BXU and the L-bus. The BXU uses 
different signals on the L-bus depending upon whether it is set in Processor or Memory mode. This 
section describes the L-bus interface when the BXU is in Processor mode. Operation in Memory 
mode is described in the "Memory Support" section of this chapter. 

The L-bus interface provides several functions listed below: 

• Direct interface between the BXU and the L-bus 

• L-bus arbitration logic 

• L-bus signal generation 

• Forming AP-bus packets from L-bus signals 

• Address recognizers that support multiple memory address ranges 

When there is more than one BXU on the L-bus, this interface coordinates the activity of the BXUs 
to provide efficient operation with multiple AP-buses. In this case, the L-bus address recognizers 
may be set up to support interleaving between multiple AP-buses. This allows the AP-bus activity 
to be shared among the AP-buses in the system. 

Memory Address Recognition 

The BXU's L-bus interface has four memory address recognizers and a special one called the Ini- 
tialization-RAM (INIT-RAM) recognizer. The four memory address recognizers allow portions of 
the system memory to be interleaved and other portions to be non-interleaved. Moreover, one of the 
four memory address recognizers maps requests to private memory SRAM on the L-bus. 

INIT-RAM Recognition 

The INIT-RAM recognizer performs two functions: it allows bootstrapping software to use the cache 
SRAM as scratch pad storage during system initialization, and it provides a means for executing a 
memory test on the cache SRAM. 

The INIT-RAM recognizer matches an address under the following conditions: 

• The value of LAD 31 is one 

• The value of LAD 30 is zero 

• The Disable-INIT-RAM bit in the LBI-Control register is set to zero (see Appendix A for the 
description of the LBI-Control register). 

• The Enable-Cache bit in the Cache-Configuration register is set to zero (see Appendix A for the 
description of the Cache-Configuration register). 
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The Initialize Ram recognizer may remain enabled in parallel with the other recognizers. If the INIT- 
RAM recognizer remains enabled, the other memory recognizers cannot map addresses to memory 
locations already covered by the INIT-RAM recognizer. If a normal memory recognizer and the 
INIT-RAM recognizer match on the same L-bus request, the operation of the BXU is 
indeterminate. 

After RESET, the INIT-RAM recognizer is the only memory address recognizer that is enabled in 
the BXU. Consequently, no memory requests from a 80960MC processor flow to the AP-bus, and 
thus, the initialization processor boot code and data must be located on the L-bus. The low-order 
memory addresses are available for PROM storage and the upper addresses are reserved for the INIT- 
RAM recognizer. 

The BXU directs addresses that match the INIT-RAM recognizer towards th e local SR AM. For these 
type s of re quests, the BXU provides two functions: the sequencing of th e RE ADY signal and the 
WDj-WD address bits; and t he mapping of the LAD 16 address bit to the WYj address bit, and the 
LAD I5 address bit to the WY address bit. This provides up to 128-Kbyte of RAM storage for use 
by the initialization software. In smaller configurations, the INIT-RAM recognizer provides four 
blocks of storage for every block that is at least 4-Kbytes. See "The Cache Directory and Control 
Logic" section of this chapter for a description of the address line interface between the BXU and 
the SRAM. 

The timing mode is controlled by the Timing-Option bits in the Cache-Configuration register. The 
signal sequencing uses the slow-read, slow-write cache timing, which is the default setting after 
RESET. 

Private Memory Recognition 

The private memory recognizer provides the ability to support SRAM on the L-bus as a normal 
memory, instead of a cache. Private memory is not accessible to other modules, and may only be 
accessed by agents on the L-bus. The BXU provides registers to establish the address range of the 
private memory (see Appendix A for the description of the private memory Match and Mask 
registers). 

The WYq-WYj bits are used by the BXU to provide an address for both private memory and cache 
memory. If cache memory is not used, the private memory can be any size up to 64-Kbytes. The 
BXU provides the control signals and the address lines for the private memory on the L-bus. 

The private memory address recognizer may overlap another L-bus memory recognizer. This is the 
only case where address recognizers are allowed to overlap. If there is a match in both the private 
memory recognizer and another memory recognizer, the private memory recognizer has priority and 
the request is handled from the private memory. 
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L-Bus and AP-Bus Conversions 

During the system operation, the 80960MC processor issues commands that may require use of the 
AP-bus. For the accesses that require the AP-bus, the BXU converts the signals on the L-bus to AP- 
bus request packets. When a reply packet is returned, the BXU converts the packet to the L-bus 
signals. This section describes special L-bus to AP-bus signal conversions. 

Byte and Double-Byte Operations 

When the 80960MC processor performs a read operation that asserts a single byte enable signal, the 
BXU issues a read-byte request on the AP-bus. Similarly, when two byte enable signals (BEj-BE , 
or BE 3 -BE 2 ) are asserted, the BXU issues a read double-byte request on the AP-bus. For each of these 
cases, the BXU uses the byte enables signals to generate ADj-ADo on the AP-bus. For all other read 
operations, the BXU issues a read-word(s) request, and the value of ADj-AD are both set to zero. 

A read byte request on the AP-bus causes the serving BXU to decode the two low-order address bits 
to determine which byte enable signals are asserted on the L-bus. Similarly, any read double-byte 
request on the AP-bus causes the serving BXU to decode AD t to determine whether BE r BE ft or 
BE 3 -BE 2 should be asserted. For the read-word(s) request, the BXU asserts all the BE 3 -BE lines. 

RMW Operations 

ARead-Modify- Write (RMW) operation is initiated on the L-bus by the le ading (falling) edge of the 
LOCK signal and terminated by the trailing (rising) edge. The LOCK line may stay low for any 
number of accesses between the start and completion of the RMW operation. This allows accesses 
by another 80960MC processor on the L-bus, as well as complex indivisible operations by a single 
processor. 



T he L-bus interface of the BXU constantly mon itors the state of the LOCK line. If the previous state 
of LOCK was not asserted (at a high level), and LOCK is asserted (at a low level) during the current 
T a cycle, and the current transaction is a re ad opera tion, then BXU converts the read operation to a 
RMW-Read request on the AP-bus. After LOCK is asserted, subsequent read or write operations 
flow through the BXU without being converted to RMW operations. 



If the previous state of LOCK was asserted, and the LOCK line is deasserted during the current T a 
cycle, and the current transaction is a write operation, then the current operation is a request for 
RMW- Write operation. The BXU inserts wait states on the L-bus until all previous requests from 
the L-bus are completed. Then, the BXU issues the RMW- Write request on the AP-bus. This ensures 
the integrity of the RMW protocol in a multiprocessor, multiple bus system. 
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The BXU defines a single, specific use of the LOCK line that is used in communicating with the AP- 
bus. Requests from a 80960MC proc essor th at are not to an address recognized by the BXU 
may use any type of sequence on the LOCK line. The operation of the BXU is not affected by 
these LOCK line sequences. 

Write-Acknowledge and No-Acknowledgement IAC Replies 

The AP-bus protocol requires a Write-Acknowledge or No- Acknowledgement reply for all I AC 
requests (see section " IAC Support of the BXU" for information on IAC requests). If the IAC is 
accepted, the serving BXU sends an write-acknowledge reply. If the message buffer is full or the 
message priority is too low, or a register IAC is invalid, the serving BXU sends a no-acknowledge- 
ment reply. The reques ting BXU relays the no-acknowledgement information back to the 80960MC 
processor by asserting BAD AC on the cycle after the last data word on the L-bus. Because the BXU 
requires either a write-acknowledge or no-acknowledgement reply, it inserts wait states on the L-bus 
until it receives the reply packet on the AP-bus. These wait states are inserted on all IAC requests. 

The 80960MC processor cannot distinguish between a no-acknowledgement reply and a bad-access 
reply. When the software program receives a no-acknowledgement reply , it typically resends an IAC 
message until it is accepted. If the receiving BXU was removed from the system because of a fault 
(described in Part III), a bad-access reply is generated. Because the 80960MC processor cannot 
distinguish between the two replies, it continues to send the IAC message indefinitely. Consequently, 
the software program should limit the number of retransmissions to a reasonably small number. 

Bad-Access Replies 

The AP-bus protocol allows request packets to time-out. For time-out errors, the requesting BXU 
issues a bad-access reply and asserts the BAD AC signal to the 80960MC processor on memory read 
requests and IAC requests. 

The requesting BXU does not issue a bad -access reply for a refused memory write request. 

When the 80960MC processor reads the location that failed the write operation, the BXU asserts the 
BAD AC signal. The bad-a ccess reply on the AP-bus is transmitted during the first bus cycle of the 
reply packet. The BAD AC signal on the L-bus is asserted on the last T d cycle of the transaction. Thus, 
the L-bus interface of the BXU provides the sequencing to complete the L-bus transaction and signal 
BAD AC if required. 

Partial Write Operations 

All write requests from the 80960MC processor that require access to the AP-bus flow through the 
requesting BXU to the AP-bus as full word write-request packets with appropriate byte marks 
asserted. The serving BXU converts the byte mark signals to BE 3 -BE signals at the memory location. 
The memory controller uses the BE 3 -BE signals to write to the appropriate byte(s). 
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Use of READY During Normal Memory Operations 

The BXU inserts one wait state for all normal memory operations during the first T d cycle. The BXU 
uses this cycle to make the address recognition. This is the only wait state in a normal write operation. 
For read requests, wait states are inserted until data returns from the AP-bus. 



Use of READY During Special Memory Operations 

The BXU inserts wait states for some special types of memory operations listed below. 

• The BXU sends data to the 80960MC processor in response to a RMW-Read request after it 
receives the entire reply packet. This ensures data integrity. 

• For RMW- Write requests, the BXU inserts wait states until all other requests on the L-bus are 
completed. 

• For write requests, the BXU inserts wait states until the write reply packet is received if the 
Sequence bit in the Match register is set to one (see Appendix A for the description of the Match 
register). 

• The BXU inserts waits states if the slow cache timing options of the Cache -Configuration 
register are selected. 

• The BXU inserts wait states for all incoming IAC operations until the reply packet is successfully 
sent on the AP-bus. 

CACHE DIRECTORY AND CONTROL LOGIC 

The cache provides high speed local storage for frequently accessed memory locations. By 
effectively storing the information locally, the cache intercepts memory references and handles them 
directly without transferring the request to the AP-bus. This action results in lower traffic on the AP- 
bus and decreased latency on the L-bus, leading to improved performance for a 80960MC processor 
on the L-bus. It also increases potential system performance by reducing each 80960MC processor's 
demand for system bus bandwidth, and thereby allowing more processors in a system. 

The BXU provides the cache directory and the control logic that implements a cache coherency 
algorithm. This algorithm ensures that 80960MC processor requests are always correctly serviced, 
even if there are multiple processors, each with its own cache sharing the same data structures. 

The data storage resides on the L-bus in external SRAM components to allow access to a large cache. 
Some external logic circuitry is required to interface the BXU's control signals to the SRAMs. This 
section describes the external circuitry and the cache operation. 

Overview of Cache Memory Operation 

The BXU provides the cache directory, coherency logic, and control signals required to implement 
a local bus cache. External SRAMs are used for data storage. This approach integrates the most 
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complex part of the cache design, while keeping the data storage external to the CPU or BXU. Thus, 
system designs can use the most advanced SRAM technology available. 

The cache memory is associated with L-bus requests. In normal operation, the cache memory is 
controlled by the BXU and is transparent to software. The CACHE signal of the 80960MC processor 
indicates to the BXU whether a request is "cacheable". 

A request is cacheable on the L-bus, if the request meets the following three conditions: 

• The request is recognized by an L-bus address recognizer 

• The CACHE line is asserted (high) during the T a cycle on the L-bus. 

• The request is not an RMW-Read or RMW-Write 

If the above conditions are met, then one of the following actions occurs: 

• A CPU read or write request is a cache hit if the requested location matches an address in the 
cache directory, and data is retrieved from the cache or written to the cache. 

• A CPU read request is a cache miss if the requested location does not match an address in the 
cache directory. Data is retrieved from the system memory, and the cache memory is updated. 

• If the CPU write request misses the cache (not an address in the cache) the cache is not updated 
and the write goes to the memory subsystem via the AP-bus. 

The operation of the cache is not dependent on the size of the data transfer and therefore can support 
partial writes. The BXU does not differentiate between a reference for data or instruction and treats 
all accesses equally. 

Basic Structure of the Cache Memory 

Figure 8-3 shows the basic structure of the cache directory contained in the BXU. The BXU supports 
multiple cache configurations depending upon the number of AP-buses. The cache directory 
structure and key cache parameters are defined in the following list. 

• A line, which consists of 16 data bytes, is the basic unit of data transferred between the cache 
and system memory. A line also is referred to as a transfer block. A cache hit or miss is 
determined on a per line basis. 

• An address block is the basic unit for cache addressing. Each address block contains the 
physical address of either eight or four contiguous lines of data (128B or 64B). 

• A validbit is associated with each line within an address block. If the line present in the cache 
is valid, then the valid bit is turned on. 

• A tag consists of the address information held in the cache directory. Since many addresses map 
to a single address block, the tag information is used to identify the exact memory locations 
that are currently associated with the address block. 
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A hit occurs when the tag of an address block matches the bus address and the desired line is 
valid. 

A way stores the tag field and valid bit. If there are multiple ways, then simultaneous 
comparisons from each way are made between the bus address and the fagr fields to determine 
if the data is in the cache data array. The BXU directory is 2- way set associative. 

A set is a grouping of address blocks consisting of one address block from each way. All 
of the address blocksin a set are simultaneously selected when a portion of the L-bus address 
is decoded into a set address. The BXU directory provides 64 sets. 
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Figure 8-3: An Example of a Cache Directory Configuration (Configuration #3) 

Note: See "Cache Configuration Register" in the appendix for a complete description of the five types 
of cache configurations supported by the BXU. 

Other key cache parameters and options are explained in the following list. 



Data Size 



Because external SRAMs are used for data storage, the BXU cache directory 
supports a range of cache sizes from 1 6-Kbytes ( 1 BXU) to 64-Kbytes (four 
BXUs on four AP-buses). 
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Replacement 



When necessary, data from a new address block in main memory replaces 
data in the cache. The replacement algorithm is pseudorandom across a set 
by toggling the selection of which way to replace on every cacheable access 
to this BXU. This pseudorandom replacement is the only replacement 
algorithm used. It always specifies the way that is used for a new cache 
entry, even if the other way is currently empty. 



BXUs are initialized in the same manner as the way is updated with a new 
address. The replacement algorithm is reset by the BXU RESET signal, and 
every time the Cache-Enable bit in the Cache-Configuration register is 
cleared. After reset, the first replacement is to way 0. The selection of which 
way to replace is toggled on every cacheable access to this BXU. 



Write Allocation 



The BXU does not perform a write allocation. When a cache miss occurs 
during a write transaction on the L-bus, the BXU does not attempt to write 
to the cache. (In a write allocation scheme, an address block is allocated 
for the write operation. Any extra data needed to fill the transfer block in 
the data array is read from the AP-bus.) 



Update Policy The update policy specifies how main memory is updated when the cache 

is written to on the L-bus. The BXU uses a write-through policy. When a 
cache hit occurs during a write transaction on the L-bus, the BXU writes to 
the cache memory as well as main memory. Main memory always holds a 
valid copy of all data locations. The write-through policy implies that the 
cache cannot off-load write requests from the traffic on the AP-bus. 



Coherency 



The cache control logic implements a coherency mechanism to guarantee 
that all 80960MC processor requests result in returning correct and consis- 
tent data in systems with multiple caches. The cache control logic con- 
stantly monitors all write requests on the AP-bus. Whenever a write is 
performed on the AP-bus to a location that is also held in the cache (i.e., a 
cache hit occurs for the particular system bus address), the line is marked 
invalid. Thus, the next time a L-bus request is made to that location, the 
BXU accesses main memory to retrieve the most recent copy of the data. 



Cache Configuration and Control 

The configuration of the cache is controlled by two registers: the LBI -Control and Cache- 
Configuration. The LBI-Control register specifies the interleaving factor and the Cache-Configu- 
ration register specifies the directory organization and timing options. 

There are two primary functions of the Cache-Configuration register: the selection of a range of 
SRAMs for the cache data array, and the selection of the timing of the external data array. These 
options can incorporate a sophisticated, high performance cache memory, or a simple, lower 
performance cache memory. Care must be exercised, however, when setting the bits of the Cache- 
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Configuration register to avoid setting an invalid combination. 

Directory 

The BXU supports a two-way set associative cache directory with 64 sets. The Cache- 
Configuration register allows the effective directory to be made smaller. The directory storage array, 
however, always remains the same size. The information stored in the directory array is the same 
for each way within every set It consists of the following items: 



• Tag. The tag field is 20 bits long and consists of the LAD 31 - LAD 12 signals. The settings in 
the Cache-Configuration register can mask some of these bits for different configurations. 

Line- Valid bits. There are eight line-validbits, one bit for each line in the address block. The 
settings in the Cache -Configuration register can mask some of these bits for different 
configurations. 

• Tag- Valid bit. This bit indicates if the tag bits are valid. If this bit is set to zero, then the tag 
and line-validbits are ignored. 

Operation With Multiple BXUs 

A single BXU supports 16-Kbytes of cache memory. When an active module uses multiple BXUs, 
the BXUs work cooperatively to provide a larger directory and addressing for larger data storage. 
The best way to view this larger directory is to consider it as having an increased number of sets. 
Thus, a cache managed by two BXUs will have a directory consisting of 128 sets instead of 64. 

In order to use multiple BXUs, the cache must be set up for interleaving. The number of BXUs must 
be the same as the interleaving factor. For example, if there is no interleaving, the cache is controlled 
by a single BXU. If two-way interleaving is used, two BXUs are required, and if four-way 
interleaving is used, four BXUs share control of the cache. 

Coherency 

The BXU guarantees that the cache data is consistent with the latest version of memory even in the 
presence of multiple caches. The coherency mechanism continuously monitors the AP-bus traffic 
for updates to locations currently in the cache directory. 

When the coherency mechanism detects a write request from another BXU or a non-cacheable write 
request from itself, the address is directed to the cache directory of the BXU. If a cache hit occurs, 
then the coherency mechanism marks that line invalid by resetting the valid bit. It is important to 
note that a line (16 bytes) is the unit of validity. Even if a write occurs to a single byte, the entire line 
is marked invalid. 

The coherency mechanism is applicable to the cache directory. It guarantees that the BXU cache 
returns the correct data for various configurations: for systems that use multiple caches, for systems 



8-19 



inter 



AP-BUS INTERFACE USING THE BXU 



where the processors designate a single data item differently (e.g., some processors designate data 
as cacheable, and others as not cacheable); or for systems that use two 80960MC processors on a 
single L-bus. 

The coherency mechanism is based upon the write-through update policy. The write-through update 
policy ensures that the system memory always has the most recent copy of all data structures. The 
cache memory attached to the BXU never contains the only valid copy of data. This action results 
in the following conditions: 

• System memory always has the most recent copy of the system data. In other words, the cache 
memory is not the only memory location that has the latest version of the data. 

• Any access that flows through to the system memory is always guaranteed to receive the latest 
copy of a data item. 

• Any time a processor updates a cache entry, it always causes a write request on the AP-bus, 
which eliminates any hidden updates. 

The coherency mechanism maintains cache data integrity if there are two 80960MC processors on 
the L-bus. Any L-bus write request that is marked as non-cacheable causes a coherency check in the 
cache. If the location is present in the cache, the line is marked invalid. This allows two processors 
to swap memory on the same L-bus. 

SRAM Interface 

The interface between the directory and control logic of the BXU and the external SRAMs consists 
of six signals. The SRAMs directly interface to the L-bus address lines and their output drivers are 
3-stated when the memory devices are not selected. 

CR The Cache Read signal indicates that the cache SRAMs should drive data 

onto the L-bus in response to a read request. This open-drain signal can be 
shared between multiple BXUs controlling a single set of RAMs. This 
sign al ca n be used to simplify the external logic that generates the CS, OE, 
and WE signals for the SRAMs. 

C W The Cache Write signal indicates that the cache SRAMs should be written 

with the data on the L-bus. This open-drain signal can be shared between 
multiple BXUs controlling a single set of RAMs. This signal can also be 
used to simplify the external logic that generates the CS , OE, and WE signals 
for the SRAMS. 



WY^WYj WAY (WY ) - provides the high order address for the SRAMs and is 

designed to di rectl y drive the SRAM address pins without the need for any 
TTL buffers. WY indicates which one of the ways in a directory set had a 
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cache hit. WAY^WYj) indicates whether the access is for the cache or for 
private memory. If private memory is not used, WYj can be programmed to 
either be at a high level or a low level when a cache hit occurs. These open- 
drain signals remain stable throughout the length of a cache access. 

WD,-WD The Word 1 and Word (WD,-WD ) signals provide the word address bits 

for the SRAMs. These two bits indicate which one of the four words within 
an address line are accessed. Because the SRAM timing is critical, the 
address information is generated one cycle early, so that an external latch 
can be used to hold the information. These open-drain signals change for 
each word of data transferred. 



The BXU asserts READY during T cycle, T. cycle, and T d cycles when data is transferred. This 
simplif ies the lat ching of the BE 3 -BE signals for the cache (even though the processor does not 
sample READY during the T a and T r cycles). 

SRAM Support Logic 

The SRAM interface of the BXU minimizes the amount of external logic required to control the 
SRAM devices, although some external logic is required to meet the tight timing constraints of the 
SRAMs. Since the data sheet specification (T 6 ) for BXU cache signals exhibit a relatively broad 
Outp ut V alid D elay external flip-flops and gates are used to achieve better resolution of the CW, 
WDj , and WD signal edges. The address and chip select logic example is shown in Figure 8-4. This 
example is referenced in the following cache performance discussions. 

Address logic 



The address lines can be latched using the ALE signal from the 80960MC processor. The WY -WYj 
signals provide two more address bits for the SRAM. (This particular example uses only WY ). 
These bits are dri ven d i rectly from the BXU because they remain constant for the duration of the 
cache access. The WDj-WD signals are provided by the BXU. These signals are conditioned by an 
external flip-flop to provide the fast signal edges required by the SRAMs. 

SRAM Chip-Select Logic 

The SRAM chip-select signals have two paths, one for a read operation and one for a write operation. 
For a read operation, all chip-select signals are derived from the BXU's CR signal. For a write 
operation, the chip-select signal assertion is dependent upon the byte enable signals and the write 
timing. The byte enable signals from the 80960MC processor are latched during the add ress state of 
the first word. The byte enables for the ne xt word o f data are latched every cycle that READY is 
asserted. For this reason the BXU asserts READY on all T a cycles, T. cycles, and T d cycles when 
data is transferred. 

In order to ensure the cache is filled properly, the byte enable latch is cleared on read requests. No 
byte enable information needs to be held during read requests because the data is always returned in 
full words and the 80960MC processor internally selects the desired data bytes. 
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Figure 8-4: Example of External Logic for Cache Interface 

SRAM Interface Signal Timings 

Figures 8-5 through 8-8 show the timing diagram for the interface signals of the design example 
shown in Figure 8-4. Figure 8-5 references the signal timing as 3/6 access timing. This notation 
means that a one- word read or write operation occurs in three clock (CLK) cycles, and a four- word 
read or write operation occurs in six clock cycles (one-wait state operation). Similarly in Figure 
8-6, a 4/10 access timing means that a one-word read or write operation occurs in four clock cycles, 
and a four- word read or write operation occurs in ten clock cycles. 
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Figure 8-5: Signal Timing for 3/6 Read Access Timing (Fast Read Mode) 
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Figure 8-6: Signal Timing for 4/10 Read Access Timing (Slow Read Mode) 
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Figure 8-7: Signal Timing for 3/6 Write Access Timing (Fast Write Mode) 




271 079-1 29B 



Figure 8*8: Signal Timing for 4/10 Write Access Timing (Slow Write Mode) 
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Cache Fill Sequence 

The cache fill sequence occurs when the 80960MC processor issues a cacheable read request that 
misses the cache. In this case, the BXU first fetches the missing cache line from memory, then returns 
the requested data to the 80960MC processor. 

Upon detecting the miss, the BXU generates the proper 16-byte request on the AP-bus. This request 
is always aligned on a 1 6-byte boundary. When the data is returned from the AP-bus, the BXU writes 
into the cache beginning at word address zero. Figures 8-9 and 8-10 show the signal sequence that 
performs the fill operation. 

Reaction to a Bad-Access Reply 

If a request receives a bad-access reply after the cache logic issues a request on the AP-bus to fill a 
line of the cache, then the events listed below occur: 

• The line of the SRAM cache is filled with invalid data. 

• The entire cache directory in the BXU receiving the bad-access reply is invalidated. 



The BXU asserts the BAD AC pin at the proper time to relay the information about the bad-access 
reply to the 80960MC processor. 
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MEMORY SUPPORT 

The BXU provides support facilities for memory controllers. The control of a memory module is 
done jointly between the BXU and a memory controller. The BXU provides the signals for the AP- 
bus interface, advanced timing information, and access to specific registers maintained by the 
memory controller. The memory controller is responsible for the detailed timing, control, and direct 
interfacing of the memory components. Normally the memory components will be DRAMS, but 
other types of memories are supported. 

Arbitration 

The BXU can act as a primary bus master or a secondary bus master, or it can ignore arbitration 
completely. The arbitration usage is defined by the LBI-Control register, which is configured by 
software during the initialization sequence. If the BXU is acting as a secondary bus master, one 
additional cycle is added to the latency for inbound (from the AP-bus) requests. 

Normal Operation Support 

When operating in the Memory mode, the BXU handles only inbound AP-bus requests. The cache 
control logic, I/O prefetch logic, and L-bus IAC support logic are disabled and the memory interface 
signals appear at the multiplexed pins (see Table 8-1). 

Read requests flow through the BXU as quickly as possible. If the BXU is idle, the request is 
transferred from the AP-bus to the L-bus in 1 .5 bus cycles. If the Fast-Reply bit in the Lock register 
is set, the data is returned on the AP-bus in 0.5 cycle after it was received on the L-bus (see Appendix 
A for the description of the Lock register). 

Write requests are fully buffered before being passed on to the L-bus. After the BXU receives an error 
free write-request packet, it initiates the write request on the L-bus. The BXU generates the AP-bus 
reply packet two clock cycles after the receipt of the last data word (and before it begins trie local bus 
write transaction). 

After the BXU starts the L-bus transaction, it completes the write request even if an error report is 
received. Thus any write request will be performed completely or not at all. If an error report occurs 
during a write operation, the write will be done twice on the L-bus. 

RMW Locks 

In Memory mode, the BXU provides two to four RMW locks with time-outs. Four locks are available 
if th e module is not interleaved with other modules, two locks if it is interleaved. The BXU maps 
AD 6 and AD 4 lines to the four RMW -Lock bits in the Lock register as shown in Table 8-4. (Note: 1 
indicates AD line is asserted.) 

When interleaving occurs, AD 4 is used as part of the address recognition for this module. Depending 
upon the value of this AD 4 , the module uses either RMW -lock bit and RMW -lock bit 2 , or RMW-lock 
bitj and RMW-lock bit 3 . 
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Table 8-4: RMW Lock Map 
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Other Memory Support Facilities 

During initialization, the memory module design may require access to information located in the 
memory controller. The BXU provides an access by mapping 16 I AC register locations (register 
address 340 H through address 37C H ) to the memory controller. 

When the BXU receives ah IAC request to these locations, it maps the request onto th e L-bus. When 
the request is issued on the L-bus, the BXU holds the Memory/Register (MEM/REG) line low during 
the T a cycle on the L-bus. This conveys to the memory co ntrolle r that the request is for its control 
registers rather than a memory location. While the MEM/REG line is low, all the BXUs on that 
L-bus turn off their L-bus recognizers in order to ensure that the IAC on the L-bus is not accepted 
by another BXU. 

During system operation, the BXU does not translate the address of a request that flows from the AP- 
bus to the L-bus. The BXU does filter the requests to allow only the requests that are destined for 
the module. However, the support of various sizes of memory arrays and different interleaving 
factors require that the memory controller perform some address translation. 

The BXU does not indicate when partial (less than a whole word) write operations occur. Thus, the 
BXU acts like a 80960MC processor on the L-bus to the memory controller. 

IAC SUPPORT OF THE BXU 

The 80960MC processor uses Interagent Communication (IAC) requests to pass messages to another 
processor and to read and write the registers of the BXU. This section shows the mechanics of how 
the IAC facility operates. 

IAC Address Recognition 

The BXU recognizes an IAC address from the L-bus or AP-bus by comparing the received address 
to address fields in its registers. The exact match condition is different for each type of IAC. 
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I AC address recognition is enabled for the L-bus if the AP -Inactive bit in the FT1 register and Faulty 
bit in the FT2 register are both set to zero. HAP -Inactive bit is set to one, but the Faulty bit is set 
to zero, then the BXU accepts I AC requests from the L-bus, but does not propagate them to the AP- 
bus (the IAC may be handled by a backup bus in fault-tolerant designs). If the Faulty field is set to 
one, then the IAC recognition is disabled. 

The match conditions for each type of IAC described in Chapter 7 are listed below. 

Local Register Request to BXUs on the L-Bus (Type 0000 B ) 

Figure 8-11 shows the address fields for the IAC type 0000B. This IAC request is used only on the 
L-bus and does not propagate onto the AP-bus. Multiple BXUs on the L-bus may recognize a single 
I AC of th is type. They coordinate their response to this IAC by using the Subsystem Busy 
(SSBUS Y) signal that is common to all the BXUs in the same module. When the SSBUS Y signal 
is asserted, the BXUs accept the requested address, but do not contin ue with th e data cycles. This 
signal coordinates the reply packets for this type of IAC request. (The SSBUS Y signal also ensures 
that the all BXUs are not busy handling IAC requests or performing prefetch operations.) 



31 24 23 1413 10 9 

llllllllllll llllJ^Ba^^J^JBIt^iaai^OlolololRlRlRlRlRlRlRlRlolol 

I II I 



REGISTER/COMMAND ADDRESS 

-IAC TYPE 

271079-132B 



Figure 8-11 : IAC Address Match Conditions for IAC Type 0000 B 

The Local Register request IAC causes one of the following actions to occur on the L-bus: 

• If the processor issues a write request, all BXUs on the L-bus perform the write request. 

• If the processor issues a read request to one of the Processor-Message registers located at 
addresses 04 H -07 H , then the BXU with the 'Message-Data-Valid bit set in the Processor- 
Priority^ register responds to the read request (see Appendix A for the description of the 
Processor-Priority Q register). 

• If the processor issues a read request to one of the Processor-Message registers located at 
addresses 0C H -0F H , then the BXU with the Message-Data-Valid bit set in the Processor- 
Priority \ register responds to the read request (see Appendix A for the description of the 
Processor-Priority \ register). 

• Unless one of the above conditions occurs, the message BXU replies to the request. AP-bus 
is the message bus unless it has failed, in which case AP-buSj becomes the message bus. The 
message BXU is defined by the settings in BXU registers: the AP-Bus-ID field of the System- 
Bus-ID register is set to 00 B , and the AP -Inactive biiof FT 1 register is set to zero; or the AP-Bus- 
ID field of System-Bus-ID register is set to 01 B and Backup-Bus-Inactive bit in the FT2 register 
is set to one (see Appendix A for the description of the System-Bus-ID register). 
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Register Request Using a Logical Address (Type 001 B ) 

Figure 8-12 shows the match conditions for IAC type 0010 B . When the 80960MC processor issues 
a logical address register request IAC, a match occurs at the BXU L-bus interface if the IAC's BB 
field (AP-bus#) matches the BXU System-Bus-ID register. This determines which BXU on the L-bus 
processes the IAC. The BXU will process the request internally if the UUUUUU field of the IAC 
matches the ID number contained in the Logical-ID register. The IAC request will propagate through 
the BXU onto the AP-bus if its Logical-ID number does not match the BXUs assigned Logical-ID 
number. 
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Figure 8-12: IAC Address Match Conditions for IAC Type 001 B 

If the address (IAC) is propagated onto an AP-bus, a match will occur at the AP-bus interface of the 
appropriate BXU if the UUUUUU field of the IAC matches the BXUs Logical-ID register. 

The Access bit is used to identify a single BXU from others within the same logical module in a fault 
tolerant system. Chapter 12 describes the usage of this bit. 

This IAC address match at the AP-bus interface is only operational if both the A/ 3 '-Inactive bit in the 
FT1 register and the Faulty bit in the FT2 register are zero. If either of these bits are a one, this IAC 
address match is disabled. 

Register Request Using a Physical Address (Type 01 00 B ) 

Figure 8-13 shows the match conditions for IAC type 0100 B . When the 80960MC processor issues 
a physical address register request IAC, a match occurs at the L-bus interface if the IAC's BB field 
(AP-bus#) matches the BXUs System-Bus-ID register. The BXU processes the request internally if 
the KCCCCC field of the IAC address matches the ID number contained in the Physical-ID register 
(see Appendix A for the description of the Physical-ID register). The IAC request will propagate 
through the BXU onto the AP bus if its Physical Bus ID number does not match the BXUs assigned 
ID number. 

If the address (IAC) is propagated onto an AP-bus, a match will occur at the AP-bus interface of the 
appropriate BXU if the KCCCCC field of the IAC matches the BXUs Physical-ID register. 



8-30 



inV 



AP-BUS INTERFACE USING THE BXU 



31 24 23 181716151413 10 9 

l1|1|1l1|1l1|1l1|K|C|C|ClC|ctefeaB|B|0|l|0|0|R|R|RlR|R|R|R|RlQ|0| 
J l , II II 



~T 



~~r 



~r 



271 079-1 34B 



REGISTER/COMMAND ADDRESS 

IAC TYPE 

MUST MATCH SYSTEM-BUS-ID REGISTER 

MUST MATCH PHYSICAL-ID REGISTER 



Figure 8-13: IAC Address Match Conditions for IAC Type 0100 B 

This IAC address match at the AP-bus interface is only operational if the AP -Inactive bit in the FT1 
register is zero. If this bit is one, then this IAC address match is disabled. This IAC is used primarily 
in fault-tolerant designs. 

Identify Device Order (Type 0111 B ) 

The BXU handles this IAC request if the IAC's BB field (AP-bus#) matches the BXUs System-Bus- 
ID register, as shown in Figure 8-14. This type of request always propagates on the AP-bus. No reply 
is generated by the BXU for these requests. See Chapter 14 for more details on the Identify Device 
Order IAC. 



31 


24 23 16151413 10 9 


I1I1I1I1I1H 


iihl I I w i | j jBiBioiihhn * i i r\ 1 rn 


271079-135 


L_ 


t 

lAf* TYPF 






SYSTEMBUS-ID 
REGISTER 



Figure 8-14: IAC Address Match Conditions for IAC Type 0111 B 

IAC Message (Type 001 1 B ) 

Figure 8-15 shows the match conditions for IAC type 001 1 B . This request type is always handled by 
the modules message BXU. The modules message BXU is the BXU attached to AP-bus Zero (unless 
a bus switch has occurred in a multiple AP-bus system). The request normally propagates onto the 
AP-bus zero (message bus), through a receiving BXU to one of two possible processors on the 
receiving BXUs L-bus. 
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Figure 8-15: I AC Address Match Conditions for I AC Type 001 1 B 

If the address (IAC) is propagated onto the AP-bus, a match will occur at the AP-bus interface of the 
appropriate BXU if the UUUUUU field of the IAC matches the BXUs logical ID register. An IAC 
message between two processors on the same local bus will not propagate onto the AP-bus. 

This IAC address match at the receiving BXUs AP-bus interface is only operational if both the AP- 
Inactive bit in the FT I register and the Faulty bit in the FT2 register are zero. If either of these bits 
are a one, this IAC address match is disabled. 

IAC Message Support 

IAC messages are used by 80960MC processors to transmit interrupts and other information on the 
L-bus and AP-bus. Because the 80960MC processors are not directly attached to the AP-bus, the 
BXUs act as the transceivers for IAC messages on behalf of the 80960MC processors in their module. 

For each of the two processors that may reside in a module, the BXU provides two registers: a 
Processor-Priority register, and a Processor-Message register that stores up to four data words. 
When an IAC message is received over the AP-bus for one of the processors on the BXU's L-bus, 
the BXU compares the priority of the message to the priority stored in the Processor-Priority register 
and checks the Message-Buffer -Full bit in the Processor-Priority register. 

Based on the priority comparison and the message buffer availability, the message packet is either 
accepted or rejected. If the message is accepted, the BXU performs the following actions: it sends 
a write-acknowledge repl y pac ket o n the AP-bus; it notifies the 8 Q96QMC processor of the pending 
message by asserting the IAC pin (IAC for Processor or IAC 2 for Processor^; and it stores the 
message in the Processor-Message register until the 80960MC processor can respond. If the 
message is rejected, the BXU sends a no-acknowledgement reply IAC. The details of this routine 
are described as follows: 

Normal Operation 

IAC messages are sent using the IAC MESSAGE (type 001 1 B ). IAC messages are read using the 
"Local Register Request to BXUs on the L-Bus" IAC (type 0000 B ). The sequence during normal 
operation is described as follows: 
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• The BXU on AP-bus acts as the message BXU. 

• The 80960MC processors on the L-bus write their current priorities into the appropriate 
Processor-Priority register using IAC type 0000 B . This updates the Processor-Priority register 
in all BXUs in the module at the same time. 

• When an IAC message is received on AP-bus , the Priority field in the IAC address is compared 
to the Priority field of the Processor-Priority register. If the IAC priority is 3 1 or is greater than 
the processor's priority, and the message buffer is empty, then the IAC message is accepted. 

If the IAC message is accepted, then three actions occur: an write-ackno wledge reply is returned 
on the AP-bus; the IAC message data is loaded into the Processor-Message registers; and the 
Message-Data-Valid bit in the Processor-Priority register is set. When the Message-Data- 
Valid bit is set, the BXU asserts the appropriate IAC signal and sets the Message-Buffer-Full bit 
in the Processor-Priority register. 

• The processor responds to the IAC signal by reading the Processor-Message registers using the 
IAC type 0000 B . The data is provided by the BXU whose associated Message-Data-Valid bit 
is set. The status bits do not change as a result of the read operation. If the Message-Data-Valid 
bit is not set by any of the BXUs, no L-bus reply to the CPU is generated and the bus activity 
is suspended (the bus hangs). 

• The 80960MC processor may change the Priority field in the Processor-Priority register while 
it is processing the IAC by using IAC type 0000 B . 

• Upon completing the IAC processing, the 80960MC processor clears the Message-Data-Valid 
bit in the Processor-Priority register. This action automatically causes the Message-Buffer-Full 
bit to be cleared. The 80960MC processor may also update the Priority field at this time by using 
IAC type 0000 B . 

• The IAC pin is guaranteed to be deasserted (high) for at least one cycle. Furthermore, the IAC 
pin is guaranteed to be deasserted by the first cycle after the completion of the L-bus request that 
cleared the Message-Data-Valid bit. 

• If the message is not accepted, then a no-acknowledgement reply is returned on the AP-bus. 
None of the status bits of the Processor-Priority register or Processor-Message registers are 
modified. 

Interaction With the Registers 

The following list shows the details of the interaction of the Processor-Priority and Processor- 
Message registers to the IAC message function. 

• The Processor-Priority Q register holds the priority of the activity that 80960MC processor is 
currently executing and the status bits for the processor's message buffer. 

• The Processor-Message^ registers (register address 010 H through address OIC H ) hold four 
words of data from an IAC message for the 80960MC processor . This register block is written 
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whenever an IAC message is accepted for processor^ The first data word of the message is 
placed in register 010 H with the subsequent words going in the next sequential register. The 
80960MG processor^ normally reads the IAC message as a single four- word burst read request. 

These registers can be read and written using individual register IAC requests. 

The status bits of the Processor-Priority Q register are only modified as a result of an IAC 
Message or an IAC write request. A normal write operation to the Processor-Message register 
does not change the status bits in the Processor-Priority register. 

• The Processor-Priority { register holds the priority of the activity that 80960MC processor is 
currently executing and the status bits for the processor's message buffer. 

• The Processor-Message ■■ registers (register address Q30 H to address 03C H ) hold four words of 
data from an IAC message for 80960MC processor . This register block is written whenever an 
IAC message is accepted for 80960MC processor x . The first data word in the message is placed 
in register 030 H with the subsequent words going in the next sequential registers. The 80960MC 
processor normally reads the IAC message as a single four- word burst read request. 

These registers can be read and written using individual register IAC requests. 

The status bits of the Processor-Priority l register are only modified as a result of an IAC 
Message or an IAC write request. A normal write operation to the Processor-Message l register 
does not change the status bits in the Processor-Priority ^ register. 

I/O PREFETCH LOGIC 

The I/O prefetch unit enhances throughput time for sequential I/O data transfers by providing two 
32-byte internal buffers. During typical operation, a memory-mapped addres s is sent t o logic 
external to the BXU. After the logic decodes the address, it asserts the Prefetch (PFETCH) signal 
and sends a Start-Channel command (a one- word write request) to the BXU. The Prefetch-Control 
register is setup to handle the prefetch function. The BXU fetches 32 bytes of data based upon the 
address of the Start-Channel command. (See Appendix A for the description of the Prefetch-Control 
register.) 

When the external logic needs a data word, it reads it from the buffer, thus saving an access to the 
AP-bus. When the external logic reads the last word in one of the channels, the BXU automatically 
retrieves another four words of data and places it in this buffer. The prefetch function provides a 
significant increase in I/O performance because the data requests are handled immediately from the 
prefetch buffers. 

Because the normal operation of the BXU hides the latency of write requests by replying immediately 
on the L-bus, the prefetch unit only operates on read requests. The BXU does not perform a 
coherency check on data residing in the I/O prefetch buffers. Consequently, the processor using an 
I/O prefetch channel must have exclusive access to the data flowing through the prefetch unit. 
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Signal Definition 



The PFETCH signal is used in conjunction with the CACHE and W/R signals to define the type of 
request being issued. Table 8-5 defines all of the different request types. 

Table 8-5: Prefetch Signal Decoding 





CACHE 
Signal 




Operation During T a Cycle 


PFETCH 
Signal 


WRITE/READ 
Signal 











Read using Prefetch Channel 








1 


Start-channel command for Prefetch Channel 





1 





Read using Prefetch Channel! 





1 


1 


Start-channel command for Prefetch Channel 


1 








Non-cacheable read 


1 





1 


Non-cacheable write 


1 


1 





Cacheable read 


1 


1 


1 


Cacheable write 



NOTE: = Low, 1 = High 



When the PFETCH signal is deasserted (high), the current cycle is a normal L-bus cycle. When the 
PFETCH pin is asserted (low), the current cycle is an I/O prefetch cycle. D uring prefe tch cycles the 
CACHE pin is used to select one of the two I/O prefetch channels. The PFETCH line must be 
connected to V cc through an external pull-up resistor when it is not used. 

Registers and Commands of the I/O Prefetch Unit 

Each prefetch unit uses two channels that consist of two 16-byte buffers each, which are shown in 
Table 8-6. The Prefetch-Control register is used to specify the channel and other options. 
This unit responds to two commands: Start-Channel and S tart-Channel r 
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Table 8-6: I/O Prefetch Unit 



Channel 


Buffer 


Address 








3C0 H -3CC H 





1 


3D0 H -3DC H 


1 





3E0h-3ECh 


1 


1 


3F0 h -3FCh 



Start-Channel Command 

The BXU receives a Start-Channel command before a prefetch channel can be used. A Start-Channel 
command is defined as a one-word (or less) write request to a prefetch channel. The address in the 
write request is used as the starting address for the prefetch channel. The data word is not used by 
the BXU and may be any value. 

When the BXU receives the Start-Channel command, the following actions occur: 

• The BXU marks both prefetch buffers in the channel as empty, and computes the starting 
address. 

• If the BXU is involved in the transfer, then the Active bit in the Prefetch-Control register is set. 
The BXU issues two prefetch requests on the AP-bus to fill the 32 bytes of data in the channel 
buffer. 

• If the memory locations are in a non-interleaved portion of memory, not every BXU in the 
module is involved in a transfer. The inactive channels in these BXUs cannot be used for another 
prefetch transfer. There are only two channels per module, even if a transfer does not use the 
channel in every BXU. 

Every BXU in the module replies to the Star t-Channel command when the actions listed above are 
completed by all the BXUs. (The SSBUSY line is used for BXU-to-BXU communication.) When 
the Start-Channel command is completed, the prefetch requests are initiated, but not necessarily 
finished. 



The combination of the Start-Channel command and the PFETCH pin eliminate problems that can 
arise from reading stale data (data stored in the buffer from a previous cycle). However, the memory 
being read by the prefetching processor must belong solely to that processor. 



Coherency checks are not done on data in the I/O prefetch buffer when the BXU prefetches data 
beyond the last byte of the last word in the channel buffer as defined in software. This data does not 
cause a coherency problem because of the following situations: 

• The Start-Channel command clears any potential stale data that may have been prefetched 
during the previous I/O prefetch sequence. 
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• By using the PFETCH signal, only the software currently using the prefetch channel receives 
data from the prefetch buffers. Even if a request is to a location already in the I/O prefetch buffer, 
the BXU accesses memory for the data unless the PFETCH signal was asserted. 

Prefetch Data Buffers 

Each channel has 32 bytes dedicated for data storage, which are divided into the upper and lower 
buffers. When the prefetch unit accesses data, it loads either the upper or lower buffer with 16-byte 
blocks (aligned on 16-byte boundaries). On a read request from the L-bus, the prefetch unit returns 
the amount of data requested by the 80960MC processor. The prefetch unit never has to cross bank 
boundaries because the 80960MC processor guarantees that it splits all memory requests that cross 
16-byte boundaries into two requests. 

Normal Operation 

This section describes the setup, startup, and data transfer for the prefetch unit. 

Setup 

Before an I/O prefetch operation begins, the Enable bit in the Prefetch-Control register must be set. 
However, the Enable bit does not need to be set every time the prefetch channel is assigned a new 
address range. 

Startup 

If the prefetch channels are not assigned statically, the software must allocate a channel to a particular 
data transfer. The BXU does not provide any support for this allocation process. 

The prefetch data must reside totally within a 256-Kbyte partition of memory. The I/O prefetch unit 
cannot increment across a 256-Kbyte boundary. This restriction eliminates the need for a second 
address match on the addresses generated by the prefetch unit. 

To start th e prefetch function, the external logic issues a Start-Channel command while it asserts the 
PFETCH signal. The prefetch unit uses the address in the Start-Channel command for the first byte 
to be prefetched. 

Data Transfer 

A valid request to the prefetch unit must meet the following conditions: 

• It must be a standard read request (no write or RMW-Read requests). 

• The length of the data request must be a byte, an aligned double-byte (half word), a word, or 
multiple words. 
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• It must be recognized by one of the L-bus address recognizers (not the private memory 
recognizer). 



• It must assert the PFETCH signal and have the CACHE signal point at the correct prefetch 
channel. 

Normally the requested data is already present in the channel's data buffer. If this is the case, the 
prefetch unit returns the data immediately using a 3/6 read access timing (one wait state). If the data 
is not in the buffer, then the request is held until the AP-bus request fills the buffer (the AP-bus request 
was issued earlier when the buffer was first emptied). 

Data remains in the buffer of the BXU until the last byte in the last word of the buffer is read. The 
BXU does not maintain the address information used during the prefetch function. When a prefetch 
request is received, the BXU uses the word address lines (LAD 3 -LAD 2 ) and size address lines 
(LADj-LAD ), and the byte enable signals (BE 3 -BE ). The byte enable signals determine which 
bytes in the current prefetch buffer are used. The BXU assumes that the request is within the current 
prefetch buffer. No additional prefetch address checks are done. 

When the last byte in a buffer is read, the prefetch unit computes the address for the next 1 6-byte block 
and issues an AP-bus read request to this location. This address is based upon the address from the 
80960MC processor's prefetch request along with the current interleaving factor specified by the 
LBI-Control register for this address range. The address sent on the AP-bus is within the memory 
recognition window of the L-bus interface because the incrementer only goes through LAD 17 and the 
memory recognizers only look at LAD 31 through LAD 18 . 

When a prefetch is started, the first data block goes into buffer , and the next goes into buffer r This 
pattern is repeated as needed throughout the transfer. 

DIAGNOSTIC SUPPORT FUNCTIONS 

The 80960MC processor or an external tester can test a BXU by sending bus requests and checking 
the BXU's reactions to these requests. The BXU's registers and storage for prefetch data can be read 
and written by IAC requests. This provides sufficient accessibility for test control in most cases. The 
BXU provides additional diagnostic support for testing in two areas: the cache directory logic, which 
is described in this section; and the fault tolerance logic, which will be discussed in Chapter 12. 

Cache Testing 

The BXU provides extensive testing facilities for the cache logic as well as the external SRAM. The 
cache tests are performed by a 80960MC processor on the L-bus for the external SRAM and the cache 
directory. 

External SRAM 

The external SRAM and its associated logic can be tested by using the INIT-RAM recognizer. The 
INIT-RAM recognizer allows a 80960MC processor to have access to the entire SRAM storage 
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space. Using this mechanism, the 80960MC processor can directly write and read all the SRAM 
locations. 

BXU Directory Logic 

The BXU supports testing the cache directory logic by providing the following facilities: 

• BXUs are initialized in the same manner as when the way is updated with a new address. The 
replacement algorithm is reset when the RESET signal is asserted and when the Cache-Enable 
bit in the Cache-Configuration register is cleared. After reset, the first replacement is to wayO. 
The selection of which way to replace alternates on every cacheable access to this BXU. 

The replacement algorithm is pseudorandom across a SET by toggling the selection of which 
way to replace on every cacheable access to this BXU. This pseudorandom replacement is the 
only replacement algorithm used. It always specifies the way that is used for a new cache entry, 
even if the other way is currently empty. 

Diagnostic software can control the cache replacement by asserting the Cache-Inhibit bit on all 
but one L-bus recognizer. In this way only those accesses that are part of the cache test are treated 
as cacheable by the BXU. 

• The BXU provides visibility into the BXU directory lookup table through the Cache-Test 
register (see Appendix A for the description of the Cache-Test register). 

L-Bus Interface Testing 

The BXU provides the Local-Bus-Test register, which allows the diagnostics to check on the type 
of recognition that was performed on the previous L-bus access (see Appendix A for the description 
of the Local-Bus-Test register). 

BXU TIMING 

This section focuses on the overall signal flow and timing of systems that use BXUs and the AP-bus. 
It provides the typical number of bus cycles required for an access to memory. However, the actual 
number of bus cycles required to access data is dependent upon the design. 

This section is divided into three major categories: 

1. Access time for memory requests 

2. Access time for cacheable requests 

3. Access time for I/O prefetch requests 
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Memory Requests 

Table 8-7 shows the access time for memory requests. Whenever the request flows onto the AP-bus, 
the access time is dependent on bus loading and access contention. Consequently, the number of bus 
cycles show the typical access for a BXU and an access for a loaded system. The notation indicates 
the number of bus cycles required for a one- word and four- word access. 

Table 8-7: Memory Request Access Time 





BXU 


Loaded 


READ 


11/14 


18/21 


WRITE 


3/6 


3/6 


RMW-READ 


11/17 


18/24 


RMW-WRITE 


3/6* 


3/6* 



NOTE: 

* Assumes all previous L-bus requests are 

completed. 
Notation: 1 word/4 word access. 

Outbound Read Requests 

Figure 8-16 shows how the signals propagate from the processor, to memory, and back to the 
processor on a clock-by-clock basis for a typical read operation. When a 80960MC processor or the 
cache logic on the L-bus issues an outbound read request, the BXU accepts the base address and the 
size of the request from the L-bus, but does not acknowledge the request until the data is returned to 
the 80960MC processor. Hence, during read operations that flow on the AP-bus, the BXU inserts 
wait states on the L-bus until the read operation is completed. 
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Figure 8-16: Typical Read Timing Diagram 
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An outbound read request is always the last request in the internal request queue of the BXU. When 
the read data arrives on the AP-bus, the data flows through the BXU onto the L-bus to improve 
performance. 

Outbound Write Requests 

Figure 8-17 shows how the signals propagate from the processor, to memory, and back to the 
processor on a clock-by -clock basis for a typical write operation. When the 80960MC processor on 
the L-bus issues an outbound write request, the BXU accepts the base address, the size of the write 
request, and the data from the L-bus. 



BUS CYCLE 

PROCESSORS 

L-BUS 

PROCESSOR'S 

BXU 


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 




A 


- 


D 


D 


D 


D 






y 


































X 
































!$ $$• 




AP-BUS 






B 


A 


D 


D 


D 


D 




R 










III IP ;$! 














MEMORY'S 
BXU 

MEMORY'S 
L-BUS 


















X 


X 














































m 
11 


$:?: 




A 


D 


D 


D 


D 
















A = ADDRESS CYCLE 






B = ARBITRATION 




D = DATA CYCLE 




R = WRITE REPLY ON AP-BUS 




X = BXU INTERNAL DELAY 




- = WAIT STATE ON L-BUS 271 079-1 38B 



Figure 8-17: Typical Write or Cache Write (Hit or Miss) Timing Diagram 

Write requests are immediately acknowledged on the L-bus, thereby releasing the bus and allowing 
the processors to proceed. A BXU can buffer up to three outbound write requests. If the write request 
encounters a bad-access error on the AP-bus, the BXU ignores the bad-access reply because the write 
was already acknowledged on the L-bus. When the 80960MC processor reads the location that failed 
the write operation, the BXU asserts the BAD AC signal. The BXU does, however, respond to all 
other errors on write operations (e.g., parity error). 

Write requests are fully buffered at the destination of the request. This means that all write operations 
are indivisible with respect to read operations, and that write operations are either completed or not 
completed at all (aborted). 

Outbound RMW-Read Requests 

Figure 8-18 shows how the signals propagate from the processor, to memory, and back to the 
processor on a clock-by-clock basis for a typical RMW-Read operation. RMW-Read requests 
respond like normal outbound read operations except for buffering the data. When data returns on 
the AP-bus, the BXU buffers the entire reply packet to ensure that it is correct before transferring 
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it on the L-bus. Thus, RMW-Read requests are indivisible and never return inconsistent data. The 
RMW-Read timing is extended if there is contention for the RMW lock. 
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Figure 8-18: Typical RMW-Read Timing Diagram 

Outbound RMW-Write Requests 

RMW- Write requests are handled differently from normal writes by the BXU. RMW-Write requests 
are held at the L-bus interface and are not accepted until all preceding outbound accesses from the 
same module have cleared the AP-bus (i.e. , completed without errors) . At that point, the RMW-Write 
request is accepted by the BXU and the L-bus is released. 

Access Time for Cacheable Requests 

The access times shown in Table 8-8 assume that the cache Timing-Option field in the Cache- 
Configuration register is set for a fast-read, fast- write timing option. 

Table 8-8: Access Time for Cacheable Requests 





BXU 


Loaded 


Read Hit 


3/6 


3/6 


Write Hit 


3/6 


3/6 


Read Miss 


15/18 


21/25 


Write Miss 


3/6 


3/6 



Notation: 1 word/4 word access. 
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The BXU provides one wait state for each cache access. There is no penalty for a miss on a write 
operation because the request is buffered inside the BXU and the AP-bus access time is hidden. 

Figure 8-19 shows the typical timing for a cache read miss. In this example, the cache must be filled 
before the data is returned. Thus, the BXU always performs a four- word access to memory on read 
misses. The bus-to-bus connection adds two cycles (one each way), and the BXU adds two cycles 
(one each way) for moving the data through the chip. 
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Figure 8-19: Typical Cache Read Miss Timing Diagram 



IAC Requests 



Table 8-9 shows the access time for IAC requests. All IAC requests are held until the AP-bus reply 
packets return to the BXU. This action allows the BXU to relay information to the 80960MC 
processor about the type of reply packet (write-acknowledge, no-acknowledgement, or bad-access). 
The access time may be different depending on whether the request is sent to a L-bus BXU, or whether 
the BXU is busy processing other requests. 

Table 8-9: IAC Request Access Time 





BXU 


Loaded 


IAC Read 


11/14 


18/21 


IAC Write 


11/14 


18/21 



Notation: 1 word/4 word access. 



I/O Prefetch Request 



The BXU contains prefetch logic for two I/O channels. Each channel consists of two 1 6-byte buffers. 
The prefetch buffers have to be setup before an I/O transfer. A prefetch read operation is issued on 
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the AP-bus whenever the last word of the previous prefetch buffer is read. Hence, a prefetch read 
operation is the only type of read operation that can be intermingled with, and followed by a write 
request in the AP-bus pipeline queue (see description of outbound read requests). 

Figure 8-20 shows the timing diagram for an I/O prefetch read operation. The BXU accesses data 
in the prefetch buffers in 4/7 access time, assuming that a typical AP-bus access is 17/20. An I/O 
prefetch channel in a single AP-bus system can transfer data at a rate of 25 -Mbytes per second. 
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Figure 8-20: Typical I/O Prefetch Read with Buffer Fill Timing Diagram 

An I/O prefetch channel in a dual AP-bus system (assuming that memory is interleaved across the 
two buses) can transfer data at a rate of 50-Mbytes every second. The increase in bandwidth is made 
possible because the prefetch unit accesses four buffers instead of the two buffers. The maximum 
L-bus bandwidth (assuming back-to-back 1 6-by te read operations with two wait states) is 3 2-Mby tes 
every second. 

Another way to look at the performance of the prefetch unit is to examine the prefetch unit's ability 
to isolate L-bus prefetch reads from the potentially long latencies that could occur on the AP-bus in 
multiprocessor configurations. Normally, one can expect approximately 15 wait states for a reply 
to an AP-bus request. The prefetch mechanism in a two bus system allows a sustainable 10-Mbyte 
transfer rate if the AP-bus latency is 100 wait states. 

SYSTEM CONFIGURATIONS 

A multiprocessor 80960MC based system is composed of a set of modules connected to system AP- 
buses. Figure 8-21 shows three types of modules: active, passive, and the combination of an active 
and passive (active/passive). These three types of modules can be used to build a multiprocessor 
system. 

The active and active/passive modules in a system are connected to all system AP-buses. Passive 
modules may be connected to a subset of the system buses. For example, if the system design used 
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four AP-buses, the active and active/passive modules would be connected to all four AP-buses; 
whereas, the passive module may connect to any or all of the AP-buses. Guidelines for the 
configurations are shown below: 

• Each module can be connected to as many as four system AP-buses. Each system bus connection 
requires an individual BXU. 

• Each active module can support up to two 80960MC processors (limited by the BXU's support 
for I AC messages). The number of other components is only limited by the electrical and 
physical constraints of the modules L-bus implementation. 

• Logical addressing allows up to 32 modules for every system, although electrical considerations 
typically limit the practical number of modules to 20 per system. 

The following sections describe the details of each module. 



;i; ACTIVE MODULE;: 



CPU 



I PASSIVE MODULE;: 



;• ACTIVE/PASSIVE 8 
MODULE 



£ ACTIVE/PASSIVE M 
MODULE M 



MEMORY AND 
CONTROLLER 



; L-BUS ;;; 



| : L-BUS L-BUS j F"HK 



■<y£:l<wim*kiil& 



•jyi'i'ivfoiiSj 



•AP-BUS, 



e=3 



I AP-BUS, 




271079-142 



Figure 8-21 : Types of Modules 



Active Module 



In the active modules, the L-bus traffic flows either on the L-bus to its resources (cache, etc.) or flows 
through the BXU on to the AP-bus. No request packets flow from the AP-bus through the BXU to 
the L-bus (all requests originate on the L-bus). 
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The BXU acts as an agent for the 80960MC processors and handles all IAC transactions on behalf 
of the proce ssor. The BXU informs the 80960MC processor of an incoming IAC message by 
asserting the IAC interrupt pin of the 80960MC processor. 

In an active module* the BXU is set to Processor mode during the initialization sequence (see LBI 
Control Register in Appendix A). When operating in this mode, the BXU is always an L-BUS 
SLAVE and consequently, does not need to arbitrate for control of the L-bus. 

Passive Module 

Passive modules are memory modules or modules with only slave I/O devices. In these modules, all 
L-bus requests originate from the 80960MC processor of an active module. Bus traffic flows from 
the 80960MC processor, through a BXU attached to the active module, onto the AP-bus, through 
another BXU attached to the passive module. For this case, the BXU that is attached to the passive 
module is set in the Memory Mode and always acts as an L-BUS MASTER. Slave I/O (peripheral 
or memory) devices can be accessed by using use memory-mapped addresses that are not in the IAC 
address space. 

Active/Passive Module 

Active/Passive are mixed modules containing processors and non-private memory, or master and 
slave I/O devices. In the Active/Passive modules, the BXU is set to the Processor Mode during the 
initialization sequence. 

This type of module has limited access to other Active/Passive modules in order to avoid potential 
deadlock situations. If a deadlock occurs, it can be detected by monitoring the time it requires to 
service a request. The BXU starts a timer when there is at least one inbound request. If the request 
is not serviced by the BXU after 16-32 cycles, the BXU declares a deadlock. 

To avoid a deadlock, the BXUs generate a reissue-reply packet for any inbound request when there 
is also an outbound request that has not cleared after some predefined number of cycles. This 
procedure together with access restrictions (see Table 8-9) avoids a deadlock. 

Maintaining request ordering in the presence of reissue replies requires that BXUs in other modules 
of the system be aware of the address space where reordering may occur. During initialization; 
software sets up the L-bus address recognizers in the BXUs that cover the range of addresses that may 
send reissue replies. When a request is received by a BXU that is assigned one of these special 
address ranges (i.e., the Sequence bit in the Match register is set), the BXU inserts waits states on 
the L-bus until the request is completed. This guarantees correct ordering of requests (multiple 
requests are not allowed). Note that ordering is guaranteed by the requesting BXU, not the BXU in 
the active/passive module. 

Table 8-10 shows the various communication paths that are allowed between module types. A 
module cannot send information to a module of its own type or a type that can send information to 
it. These restrictions allow the reissue reply to break a deadlock. 
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Table 8-10: Access Restrictions 



Source 
Module 


Destination Module 


Active 


Passive 


Active & 
Passive 


Active 


BXU 


Yes 


Yes* 


Passive 


— 


— 


— 


Active/ 
Passive 


BXU 


Yes 


BXU 



NOTES: 

BXU — Access restricted to lACs (captured by BXU). 
Yes — Memory accesses and lACs allowed. 
Yes* — Special setup in requesting BXUs to maintain 
access order 

There is a possibility that bus starvation can occur, however. To avoid starvation conditions and the 
performance degradation of reissue replies, the system configuration and software usage of the 
configuration should be structured to avoid frequent occurrence of these access conflicts. 



SUMMARY 

The BXU component provides the interface between the L-bus and AP-bus and makes possible high 
performance, modular system configurations. It supports a L-bus cache memory, I AC message 
passing, I/O prefetching, memory, and fault tolerance. These functions are generated by the seven 
BXU logic blocks listed below: 

AP-bus Interface 

L-Bus Interface 

Cache Directory and Control Logic 

Memory Support Logic 

IAC Support Logic 

I/O Prefetch Logic 

Fault Tolerance Support (described in Part III of this manual) 

The BXU provides two modes of operation: Processor mode or Memory mode. In Processor mode, 
the BXU acts either as a L-bus master or slave, and supports the cache, prefetch, and IAC message 
functions. In Memory mode, the BXU acts as a L-bus master and provides support for memory 
functions. 

To provide a flexible interface to the systems software, the BXU has various programmable registers. 
In addition, the BXU contains diagnostics for testing portions of the Cache Directory and Control 
Logic. 
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CHAPTER 9 
MEMORY AND I/O INTERFACE USING THE BXU 

The M82965 BXU provides many features that enhance high-performance multiprocessor designs. 
This chapter outlines approaches to design memory and I/O systems in a multiprocessor desi^ri that 
uses the BXU as the interface between the L-bus and AP-bus. The example design uses passive and 
active/passive modules, which allow systems to be built and expanded in modularity. Because these 
modules contain an L-bus, the memory and I/O interfaces previously described in Chapters 4 and 5 
are directly applicable. 

BASIC MEMORY INTERFACE 

In a multiprocessor design, the 80960MC processor communicates with the system memory over the 
AP-bus. As shown in Figure 9-1, the active module, which contains the 80960MC processor, 
transmits or receives data to or from the passive module, which contains the sy stern memory and 
controller. The BXU that interfaces to the system memory and controller is set to Memory mode and 
acts as a bus master. Because the L-bus is used, the memory controller discussed in Chapter 4 is 
directly applicable. 



ACTIVE MODULE 

iiiiiiiiiiiiiiiiili' 



PASSIVE MODULE 




AP-BUS 
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Figure 9-1 : Memory Interface Logic to the BXU 

Figure 9-2 shows the major logic blocks of the memory interface circuit. The data transceivers buffer 
the data to compensate for any slow devices that may be connected to the BXU. The address latches 
demultiplex the address/data signals from the BXU and latch the address. The address decoder 
selects the appropriate memory device from the latched address. To accommodate a memory burst 
transaction, the burst logic decrements the word count, increments LAD 3 and LAD 2 , and generates 
a CYCLE-IN-PROGRESS signal. The timing control generates a READY signal and other specific 
signals required by a SRAM. The byte enable latch stores the byte enable signals. 

The DRAM controller multiplexes the address into a row and column address, performs the refresh 
operation, arbitrates between a refresh request and memory request, and generates the necessary 
control signals for the DRAM. The SRAM interface conditions the control signals for the SRAM. 
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This memory interface is identical to the one discussed in Chapter 4 because the BXU generates the 
L-bus signal^. See Chapter 4 fpr details concerning the^ description and timing of the major logic 
blocks. ■ ' "" :% "" " 

j/OllSITERFAQE 

In a multiprocessor design, the 80960MC processor also communicates with the peripheral devices 
over the AP-bus. As shown in Figure '9-3, the active module, which contains the 80960MC processor, 
transmits or receives data to or from the passive module, which contains slave-type I/O devices,or 
an active/passive module, which contains a master I/O device and slave I/O device. 

The BXU that interfaces to the passive module is set to Memory mode and act§ as a bus master. 
Because the L-bus is used, the general I/O interface circuit discussed in Chapter 5 is directly 
applicable.-'"' ■ ■ ''•■■■■, ■■''■'••" 

The BXU that Interfaces to an active/passive module is set to the Processor mode and acts as both 
a t-biiS rhasfef arid slave. Wheti the BXU acts as a bus slave, the module must contairt logic that can 
issiie command^ to the BXU. This logic can be comprised of discrete circuits or it can be a 80960MC 
processor. 
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Figure 9-3: Overview of the I/O Interface 

General I/O Interface for a Passive Module 

The BXU can be used to interface to slave I/O devices using the L-bus. In a typical system design, 
a number of slave I/O devices can be controlled through a general system interface. 

Figure 9-4 shows the major logic blocks of a general system I/O interface. Standard 8-bit data 
transceivers add drive capability, provide bus isolation, and prevent bus conflicts that may occur with 
slow I/O components. The address latch demultiplexes the address/data lines and holds the address 
stable throughout the L-bus transaction. The address decoder generate s the I/O chip-select signals 
from the latched address lines. The timing control block provides the READY signal to the BXU 
and the I/O read and I/O write command. 

The BXU that interfaces to the passive module is set to Memory mode and acts as a bus master. 
Because the L-bus is used, the general I/O interface circuit described in Chapter 5 is directly 
applicable, except for the interrupt signals. The handling of interrupts is described in a later portion 
of this chapter utilizing the M8259A Interrupt Controller. 
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Figure 9-4: Simplified General System Interface 

I/O Interface for an Active/Passive Module 

The L-bus signals must be generated for an active/passive module when data is sent to another active 
module containing a 80960MC processor. L-bus signal generation can be derived in several ways 
depending on the type of peripheral I/O device that is used. 

This function can be performed by a 80960MC processor. However, if the computational power of 
a 80960MC processor is not needed, a simple controller comprised of standard logic circuits can be 
designed to generate the specific control signals required for L-bus operation. 
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Overview of an Active/Passive Module With the M8259A 

This section describes a design that generates the control signals in response to an interrupt request 
from a M8259A Peripheral Interrupt Controller. 

In a multiprocessor design using the AP-bus, interrupts can be hardwired directly to the 80960MC 
processor, or they can be sent to a 80960MC processor over the AP-bus by using interrupt IACs. 

Connecting interrupt signals directly to the 80960MC processor, eliminates the additional logic 
required to generate IACs. It does, however, limit interrupts to a single 80960MC processor. By 
generating IACs on the L-bus, interrupts can be processed by any 80960MC processor in the system 
designated by software. The design, which is described later in this section, requires the logic on the 
I/O subsystem L-bus to accept interrupts and generate the associated IAC messages. 

The "Message" IAC (type 001 1 B ) can be used to convey an interrupt on the AP-bus. Figure 9-5 shows 
the address format for this IAC type. The module destination field for this IAC defines which 
80960MC processor receives the interrupt message. The internal destination field of the IAC 
provides the priority of the message. 









IAC ADDRESS FIELD 
IAC ADDRESS BIT 


IAC IDENTIFICATION 


MODULE DESTINATION 


LBD 1 


IAC TYPE 


INTERNAL DESTINATION 




,|i|,|,|i|i|,|i 


U|u|u|u|u|U|N|i 


*H8 


| | 1 | 1 


g$p|p|p|p|p|o|o|o|o 


31 24 23 16 15 14 13 10 9 

NOTE: 

1. L-BUS DESTINATION 
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Figure 9-5: Address Format for the Interrupt IAC 

Figure 9-6 shows the data word for IAC type 001 l' B . The eight high-order bits designate an interrupt 
IAC; the next eight bits contain the interrupt vector; and the lower-order bits are not used. 





|0|1 


|o|o|o|o|o|o|v|v|v|v 


v|v|v|v^^^^^^^^^^ 


31 


24 23 


16 15 

271 079-1 48B 



Figure 9-6: Data Word for the Interrupt IAC 

When the destination module BXU receives an IAC for one of its 80960MC processors, the BXU 
checks the message priority (the value in the internal destination field). If the priority value is 3 1 , 
or greater than the priority of the currently executing process, then the 80960MC processor's IAC 
pin is asserted and the IAC message is read. If the priority is less than that of the current process, or 
if the BXU's IAC message buffers are full, then the BXU sends a no-acknowledgement reply packet 
on the AP-bus. 

When the interrupt IAC is accepted by the 80960MC processor, the implicit priority of the interrupt 
vector is also compared with the priority of the currently executing process. If it is higher than the 
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current process or 3 1 , then the interrupt is immediately serviced. If not, then the interrupt is posted 
to be serviced later. 

The IAC Generator 

The IAC Generator is a block of logic that interconnects an interrupt requesting source to the I/O 
modules L-bus. The IAC Generator accepts interrupt requests from an external interrupt controller, 
such as the M8259A Peripheral Interrupt Controller and produces an appropriate interrupt IAC 
message. The BXU subsequently transmits the IAC message over the AP-bus to its ultimate 
destination. The IAC generator performs four distinct operations: 

• It generates an interrupt IAC, which is a one-word write operation to the local BXU. To 
determine whether the interrupt IAC is accepted, the IAC Generator monitors the BAD AC 
signal, and resends the interrupt IAC if it is not accepted. 



• It performs an interrupt acknowledge sequence by generating two INTA pulses. The two pulses 
are used to fetch the interrupt vector after receiving an interrupt request from the M8259A. 

• It allows 80960MC processors to address the internal registers of the M8259 A for initialization 
and programming. 

• It allows 80960MC processors to read and write to the IAC message storage area. Thus, IACs 
can be specified by software. 

To perform the first function, the IAC Generator acts as an L-bus master. It initiates transfers and 
allows the BXU to control the data rate with the READY signal. The IAC Generator acts as an 
L-bus slave to implement the last two functions. Consequently, the IAC Generator and BXU arbitrate 
for the L-bus by using the HOLDR and HOLDAR signals. 

Interconnection 

Figure 9-7 shows the minimum set of si gnals required to i nterfa ce an IAC Generator to the BXU. 
Since the I AC ge nerator is synchronous, ADS can replace ALE to keep the number of signals to a 
minimum. (ADS is an input to the 82965 when the BXU is the L-Bus Slave.) The BE 3 -BE signals 
are not used by the IAC Generator if the design assumes that all transfers to/from the BXU are four- 
byte operations. This is a reasonable assumption because non-IAC transfers usually only initialize 
the IAC Generator and Peripheral Interrupt Controller. 

The IAC Generator uses two signals, RESET and SELECT (SEL), for overall control. The RESET 
signal is used to force the IAC Generator to a known state and to disabl e int errupts. Thus, the IAC 
Generator does not produce AP-bus traffic prior to initialization. The SEL signal can be used to 
distinguish the operation for the IAC Generator from other L-bus operations. 

The IAC Generator uses the standard interface to the M8259 A, except for the INT signal. This signal 
is passed through a synchronization stage to avoid a metastable state. 
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Figure 9-7: Block Diagram for Interface Circuit 



Timing 



Figur e 9-8 shows the signal timing for the interrupt IAC messages. The IAC Generator asserts the 
ADS signal and drives an address onto the LAD lines. After one bus cloc k (CLK); it drives the data 
word of the IAC message on the LAD lines. The BXU withholds READY until an write- 
acknowle dgement reply is received on the AP-bus. If a no-acknowledgement reply is received, then 
BADAC is asserted on the next cycle. 
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Figure 9-8: Timing Diagram for a Single-Word IAC Message 
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The BXU uses three-quarter cycle timing, similar to that used on the AP-bus, when it acts as a bus 
slave. For a write operation, the address or data is driven on the LAD lines by the bus master at the 
beginning of a bus clock (the A edge) and sampled by the BXU at the end of the bus cycle (the D edge, 
three-fourths of the way through the cycle in which READY is returned). For a read operation, the 
BXU drives the data on the B edge, and expects it to be sampled on the following A edge (assuming 
the data is ready). This signaling protocol allocates one fourth of a bus clock cycle for hold time and 
clock skew. 



After the IAC Generator receives an interrupt request, it sends two INTA signals to the M8259A. 
Upon receiving the second INTA the M8259A drives an interrupt vector onto its data pins. The IAC 
generator latches this vector and uses it to select the appropriate IAC message (i.e., use the vector 
as an index into IAC message memory). The bus timing of the M8259A is slow compared to the 
timing of the BXU. Thus, a state machine running at the same bus clock frequency as the BXU inserts 
delays to properly interface to the M8259A signals. 

IAC Generator Design 

The IAC Generator design is shown in Figure 9-9. It consists of six major logic blocks: the RAM 
array, state machine, interface logic, counter, address incrementer and latch, and address decoder. 

The IAC messages are stored in a 32-bit wide RAM array. The depth of the RAM is a function of 
the number of unique messages to be stored. Normally, two words are stored for each interrupt IAC. 
The state machine sequences the control signals, and the interface logic generates the control signals 
that are routed to the BXU and the M8259A. The counter provides the necessary signal delays for 
interfacing to the M8259A. 

The address incrementer and latch performs multiple functions: during interrupt acknowledge 
sequences, it latches the vector supplied by the M8259A; during IAC messages, it supplies the 
address to the IAC message RAM and increments this address for successive word accesses. In 
addition, this logic latches the address supplied by the BXU during IAC message memory accesses, 
M8259A register accesses, and memory-mapped commands. 

The address decoder generates the chip select signals for the RAM and M8259A and decodes 
memory-mapped commands received from the 80960MC processors. The 80960MC processor uses 
two memory-mapped commands: a mask interrupt request, and an unmask interrupt request. The 
mask interrupt request is used to ensure that interrupts are not generated during initialization. During 
RESET, the interrupt requests must not be sent to prevent spurious IAC messages prior to BXU 
initialization. The mask interrupt request command also allows testing of the unmask command and 
enables the IAC Generator to write to RAM. The unmask interrupt command enables interrupt IACs 
after system initialization. 
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Figure 9-9: Block diagram for IAC Generator 

State Diagram 

Figure 9-10 shows a state transition diagram for the state machine. Table 9-1 defines each state. 
This design assumes the following conditions: 

• The RESET signal forces a transition to the idle state (I). 

• The state machine is the primary bus master (other devices, including the BXU, must assert 
HOLDR to acquire the bus). 

• An additional bit is available to store the interrupt mask bit. 
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Figure 9-10: State Diagram for I AC Generator 
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Table 9-1 : State Functions 


State 


Name 


Signals 
Asserted 


Other Action 


Comments 


I 


Idle 


None 


Set MASK bit after 
reset 


Waiting for INT from M8259A, 
or HOLDR from BXU 


A 


Addresso 


HOLDAR 


Latch LAD 4 -LAdo if 
ADS is asserted 


Qrant bus to BXU; place 
ADS in high impedance 


A, 


Address, 


HOLDAR 


Decode latched 
address 


Wait here if SEL not asserted 


DA 


Data Acce^So 


HOLDAR, RD, 
orWR 


Start delay timer 


Wait for DLY to met M8259A 
TRLRHorTWLWHspec 


DA, 


Data Access, 


HOLDAR, RD 
orWR 
READY 


Set or reset MASK 
bit if commanded 


Latch data at end of cycle if 
write operation; assert 
READY to complete transfer 


BW 


fusWait 


HOLDAR 


None 


Wait for BXU to release L-Bus 


IM 


IAC Message 
Address 


AGS, RD v 


Increment IAC RAM 

address at end of 

cycle 


Send IAC address; place 
READY in high impedance 


IM, 


IAC Message 
Data Word, 


RD 


Increment IAC RAtyl 

address at end of 

cycle 


Send Data Word,; increment 
address Modulo 2 


CK 


Check for 
BADAC * 


None 


None 




If BADAC asserted retry IAC 
message 


IA 


Interrupt 
Acknowledge \ 


INTA 


None 


Wait for DLY to meet M8259A 
TRLRHspec 


II 


Interrupt Idle 
\ .. 4 Time 


None 


None 


Wait for DLY to meet M8259A 
TRHRLspec 


VF 


Interrpt 
Vector Fetcho 


INTA 


*Latch shifjed 

vector at end of 

cycle 


Wait for DLY to meet M8259A 
TRLRH spec 


VF, 


Interrupt 
Vector Fetch, 


fNTA 


None 


Interrupt vector decode 
wait 



NOTE: 

* Since each interrupt IAC message occupies two words in RAM, the vector must be shifted left by 1 bit in 
i order to serve as a valid IAC message address. 
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Address state (A ) and Address state 1 (A,) allow the BXU or another bus master to arbitrate for 
the L-bus and check that the current access is for the IAC Generator. Data Access state (DA ) and 
Data Access state 1 (DA,) generate the timing for accessing the internal registers of the M8259A or 
IAC message RAM. During the Bus Wait state (BW), the IAC Generator gains control of the L-bus 
again. 

The interrupt acknowledge operation and IAC message transmission are performed by the following 
state sequence: the Interrupt Acknowledge state (IA), the Interrupt Idle state (II), the Vector Fetch 
state (VF ), the Vector Fetch 1 state (VF,), the IAC Message state (IM ), the IAC Message 1 state 
(IM,), and the Check state (CK). 

The values of the signals are a function of current state except for RD and WR. These signals are 
conditioned by the W/R output of the BXU, when it controls the L-bus. 

SUMMARY 

A multiprocessor system can be designed by using active, passive, and active/passive modules. Each 
module contains at least one BXU, which provides an L-bus to AP-bus interface. 

A passive module can be used to supply the system memory. In this case, the BXU is set to Memory 
mode and generates the L-bus signals. The memory interface circuit described in Chapter 4 can be 
used. 

A passive module can be used to communicate with slave I/O devices. In this case, the BXU is also 
set to Memory mode and generates the L-bus signals. The general interface circuit described in 
Chapter 5 can be used. 

An active/passive module can be used to communicate with slave I/O devices and master I/O devices. 
In this case, the BXU acts as a bus master and a bus slave, and, consequently, arbitrates for the L- 
bus. When the BXU acts as a bus slave, a controller is required to send signals to the BXU, which 
passes them to the AP-bus. This controller can be a 80960MC processor or a circuit comprised of 
standard logic devices. For example, an IAC Generator designed for the M8259A, was illustrated in 
this chapter. 
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CHAPTER 10 
FUNDAMENTAL CONCEPTS OF FAULT HANDLING 

A fault handling cycle consists of four phases: error detection, error confinement, error reporting, and 
recovery. During the detection and confinement phases, hardware errors and software bugs are 
detected and isolated to an area with a tightly defined interface. These faults are reported throughout 
the system during the error reporting phase. Finally, recovery mechanisms mask the effects of the 
fault from the rest of the system and if possible repair the faulty subsystem. 

This chapter introduces a model for fault handling based upon these four different phases and shows 
the implementation of this model in the 80960 hardware system. 

A MODEL FOR FAULT HANDLING 

A system is generally made up of a hierarchy of fault handling cycles. Although the definition of the 
levels varies depending on system design and the technology of implementation, most current 
systems contain the following five levels in ascending hierarchial order. 

1 . Component Level. This is the lowest level of the hardware system. Failures occur within the 
component itself, such as the loss of a memory bit within a RAM chip. This failure could be 
detected by employing checksums across the internal rows of storage cells in the chip. Recovery 
might be implemented by an on-board associative memory, which provides back-up storage for 
bad cells in the normal array. By employing fault handling strategies, such as checksum and 
back-up memory, higher levels in the system may be totally unaware that a fault occurred and 
was corrected. 

2. Module Level. This level is composed of a group of components (such as an 80960MC 
processor and BXU). The failures at this level occur in the components or the signal paths 
connecting the components. An example of a failure at this level would be the loss of a memory 
bit in a 32-bit word of a memory array (e.g., a bad solder joint at a RAM data-out pin). This 
failure could be detected by employing parity to cover single failures within a memory word. 
Recovery could be accomplished by replacing the parity bit with a single error-correcting 
Hamming code (ECC). 

3. Hardware System Level. This level consists of a network of interconnected modules. Failures 
may occur when either a module or the interconnect network fails. An incorrect response from 
the memory module is an example of a failure at the hardware system level. This failure could 
be detected by duplicating the memory module and comparing the outputs. Recovery is possible 
if a redundant memory module is available to replace the faulty module. 

4. System Software Level. The operating system is responsible for providing expanded service 
to the application software. Failures may occur when defects exist in the algorithms. For 
example, a software algorithm may allow a deadlock situation to occur. This can be detected 
by utilizing some secondary routines which monitor the operation of the system resources. 
Recovery might be accomplished by changing some of the previous resource allocation 
decisions. 
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5. Application Level. This level completes the system description. This software may have 
defects in its algorithms or specif icatibns. The human interface of the system may even be used 
to detect and recover from application failures. 
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Figure 10-1: Fault Handling Model 

Assuming that these system levels are connected, failures that occur at one level may flow up to 
higher levels if either the detection or recovery at that level is inadequate to handle the fault (see 
Figure 10-1 ). Each time a failure is reflected up, the detection mechanisms at the next level will treat 
the failure as if it were originally generated at the higher level. This treatment causes a loss of 
information about the fault by potentially masking the true cause of the failure, and by increasing 
the amount of the system which must be considered suspect. 

An example of fault detection being reflected up is an undetected memory error that appears as an 
invalid data structure at one of the software levels. An example of recovery being reflected to higher 
levels is an uncorrectable memory error in an application program segment. It may be possible for 
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the operating system or the user to handle this fault. Recovery can also be reflected up because of 
improper detection or incorrect operation of the recovery mechanisms. An example of this is a 3- 
bit memory error that is detected as a single-bit error by the Hamming code, and is subsequently 
corrected. 

From the preceding model, several key points about the system fault handling can be made, as listed 
below: 

• Faults can occur at many different levels in the system. 

• Each level has its own characteristics, and different detection and recovery strategies are appro- 
priate for different levels. 

• Faults not handled at one level propagate up to the higher levels in the system. 

• Higher levels have more complex environments, which make recovery a more complex and 
slower task. 

As failure modes increase in complexity, the interaction between subsystems grows, and the original 
source of the failure becomes more ambiguous. Thus, faults should be handled at the lowest possible 
level. 

80960 FAULT HANDLING APPROACH 

The 80960 fault handling approach provides general-purpose, adaptable, and software-transparent 
fault-tolerant capabilities. The 80960 system design operates under two fundamental principles. 
First, the number of hardware errors which are reflected up to the software levels of the system are 
kept to a minimum. Second, all of the fault handling mechanisms are independent and orthogonal. 

Figure 10-2 shows the location of the barrier that prevents hardware failures from being reflected into 
higher layers of the system. It is important to note that it is impossible for any system to detect and 
recover from all failures that might occur. However, the 80960 hardware reduces the rate of failures 
reflected into the software to an extremely low level. 

The reflection of errors is minimized to prevent information overload at higher levels in the system 
structure. If all failures are allowed to propagate to the top, the system becomes overloaded and loses 
its ability to react to the fault conditions. The complexity of the failure modes may make 
implementation impossible or force a reduction in the completeness of fault coverage. 

By performing detection and recovery from hardware failures at the lowest practical level in the 
system, a more general and complete solution is possible. This approach divides the responsibilities 
for fault tolerance, allowing faster solutions to fault detection and fault recovery. The mechanisms 
for detection and recovery from software errors need only address the set of faults that can be 
generated at those levels. 

By controlling and reducing the amount of errors reflected up to the next level, parallel and 
independent development may proceed on different levels (hardware, system software, applica- 
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tions). The designers at one level can safely assume that lower levels always provide consistent and 
correct operation. 
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Figure 10-2: The 80960 Architecture's Separation of Hardware and Software Layers 

All of the fault handling mechanisms used in the 80960 system are orthogonal. Expansion of bus 
bandwidth, logical resources, detection capabilities, or redundancy may be done without any side 
effects on the rest of the system. Minimizing the interaction between these variables provides the 
system with a flexible and modular basis for growth and adaptation to the application environment. 
System capabilities may be added or removed without affecting the application software. There is 
no penalty in performance, cost or system size for those fault- tolerant mechanisms not used in a 
system. 

Although all of the hardware fault handling occurs without software assistance, software remains 
responsible for managing the overall fault-tolerant policy. This division of labor allows the software 
to tailor the resources of a system to the needs of an application without giving up any of the benefits 
derived from placing the fault handling mechanisms in the hardware. Software may also periodically 
test the detection and recovery mechanisms while the system is on-line. This allows any latent errors 
in the system to be uncovered before another error would force the system to face a double-failure 
condition. 
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FAULT TOLERANCE IMPLEMENTATION FOR AN 80960 SYSTEM 

The 80960 architecture uses VLSI replication as a basis for implementing fault-tolerant designs. By 
VLSI replication, an 80960 fault-tolerant system can be designed that confines and reports errors, 
and recovers from failures. 

VLSI Replication 

The replication of VLSI components is the basic principle that forms the foundation for the 
implementation of the 80960 fault handling mechanisms. The 80960 family offers a wide range of 
system configurations from a small set of VLSI components. The same components provide modular 
expansion of performance, memory storage, detection, and recovery capabilities. Through VLSI 
replication, software is only responsible for initialization, not fault detection, isolation, or recovery. 
There are no special-purpose components aimed solely at fault-tolerant functions. 

Figure 10-3 illustrates how these components can be configured. For basic fault tolerance operation, 
VLSI components can be used for self-test functions, parity, and retry operations. 

These components can be logically paired to form self -checking modules using a technique to detect 
errors called Functional Redundancy Checking (FRC). With FRC, software designates one module 
of the pair as the master and the other as the checker. The master and checker operate in lockstep. 
The master generates the AP-bus requests, the checker compares all information sent to an AP-bus 
by the master. If either the master or the checker detect a disagreement, it will generate an error report. 
This is an example of module level error detection. 

This master-checker pair can be replicated to form a Quad Modular Redundant (QMR) Primary- 
Shadow pair. In this configuration, every FRC pair in the system is married with another self- 
checking FRC pair of the same type. This QMR pair of self-checking modules operates in lockstep 
and provides a complete and current backup for all state information in the module. QMR is also 
known as module shadowing because if one module (the primary) fails, a duplicate module (the 
shadow) is ready to operate. This QMR scheme implements the 80960 architectural mechanism in 
hardware. 

By using BXUs, all communication in the 80960 system is done over the Advanced Processor or AP- 
bus. This makes orderly growth possible since no signal definition is dependent on the number of 
resources in the system. This approach also makes on-line repair possible. The presence or absence 
of any module cannot prevent communication between any other modules. The AP-bus provides a 
uniform and regularly structured communications path that supports the modular expansion of both 
fault-tolerant and standard multiprocessor system capabilities. 

An 80960 system configured for fault tolerance will utilize at least two AP-Buses, and can have a 
maximum of four AP-Buses. (One, two, or four AP-bus configurations are allowed in general. Three 
is not allowed.) Unlike the FRC and QMR configurations, the replicated AP-buses do not duplicate 
transactions. Instead different transactions occur on each AP-bus and the AP-bus band width 
doubles. This still allows the system to recover from the failure of an AP-bus by retrying and 
redirecting bus traffic over the surviving bus. 
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Figure 10-3: VLSI Replication 



Confinement and Detection 



A confinement area is defined as a unit (module or AP-bus) of the system which has a limited number 
of tightly controlled interfaces. The confinement area limits the damage from error propagation and 
localizes the faulty area for recovery and repair. 
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Detection mechanisms (BXUs) are placed at every interface to ensure that no inconsistent data can 
leave the area and corrupt other confinement areas. When an error occurs in the system, it is 
immediately isolated to a confinement area. The error is known to be in that confinement area, and 
all other confinement areas are known to be error-free. 

By defining confinement areas, a conceptual framework is developed for the systematic and coherent 
placement and definition of the detection mechanisms. The confinement areas also provide a 
conceptual view of the system under fault conditions. This clarifies the external (software) view of 
the hardware and eliminates the need for diagnostic probing as a method of fault isolation. 

The Reporting and Recovery Cycle 

Once an error is detected and confined to an area the error is reported to other confinement areas and 
recovery can begin. The reporting and recovery sequence for an 80960 system consists of six steps: 

1. Error Reporting 

2. BXU Partner Communication 

3. Transient Waiting Period 

4. Retry 

5. Recovery 

6. Probation 

Figure 10-4 shows the state diagram for the error reporting cycle. After a BXU detects an error, it 
issues a report. When the other BXU's receive the error report, they stop their bus activity. The 
reporting phase lasts 256 AP-bus clock cycles from the beginning of the first report. The same 
reporting sequence is used independent of system configuration. 

During partner communications, the paired BXUs on two separate AP-buses exchange information 
about the state of the outstanding requests on the failed AP-bus. This action allows the BXU on the 
surviving bus to correctly retry any requests that were still outstanding on the failed bus. 

Errors can be of two types: permanent or transient. If a permanent error is detected, the recovery 
stage is started. During the recovery stage, the BXUs automatically reconfigure themselves. After 
recovery the BXUs enter the transient wait state. If a transient error is detected, the BXUs directly 
enter the transient wait state. 

During the transient waiting period, the BXUs are quiescent for 256 cycles on the AP-bus to allow 
system transients to settle before resuming normal system operation. This approach allows the 
hardware to correctly react to transient errors that are of a relatively long duration (mechanical 
vibration or motor startup). The transient waiting period can be extended by an adjustable timer that 
is set by using the Maxtime register (see Appendix A for the description of the Maxtime register). 

Following the transient waiting period the hardware system enters the retry state. During this state, 
all accesses that were pending at the time of the error report are retried. A BXU does not accept any 
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new requests until it has retried all of its previously outstanding requests. Each BXU retries its 
requests in the same order that it received them from the L-bus. The traffic on the AP-bus, however, 
may not necessarily be retried in the same order as it was before the error report. During the retry 
interval, the fault handling logic checks the bus activity for the occurrence of an error. If a second 
error is detected, a start bit is issued to begin another error reporting cycle. 



NOTERMINATE- 
PERUANENT-ERROR- 
WINDOW COMMAND 




256 CYCLES 
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Figure 10-4: The Reporting and Recovery State Diagram 
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A recurring transient error is considered a permanent error if it occurs again during the probation 
period (called the permanent error window). The time period starts when the retry state begins. The 
permanent error window ends when the CPU issues a Terminate-Permanent-Err vr '-Window com- 
mand (controlled by software) after the retry state. The BXU propagates the Terminate-Permanent- 
Error-Window command. If a transient error occurs within this software defined time period, the 
BXU issues a start bit to begin the error reporting cycle. The BXU marks the error as permanent if 
it is the same error that previously occurred. 

Fault Recovery Configuration Examples 

Figure 10-5 illustrates the range of fault tolerant alternatives available to 80960 system designers. 
The 80960 architecture supports fault recovery through retry algorithms and reconfiguration 
facilities in the BXU with the replication of CPUs, BXUs, and RAM arrays. How much replication 
is used depends on the level of fault tolerance desired for a particular application. 

Fault tolerance can range from a simplistic single module implementation utilizing software 
reconfiguration to a full QMR system with transparent hardware reconfiguration. 

Basic Configuration 

Figure 1 0-6 shows an example of a system design that provides a rudimentary level of fault tolerance. 
This system has the ability to detect parity, bus arbitration, and time-out errors on the AP-bus. 

Within this system, several techniques are used for error recovery. Parity errors are handled by the 
BXU using retry algorithms, which signal the agent that sent the faulty transaction to resend it. Time- 
out errors are handled by a fault handling routine executed by the processor. Recovery may not be 
possible if the time-out error is the result of faulty software. 

Recovery from errors at this fault tolerance level is limited to transient errors. However, since 
transient errors account for the majority of failures in a system, this basic fault-tolerant system 
provides a high level of data integrity and reduces the system downtime due to transient conditions. 

Errors resulting from hardware failures would require system shutdown and replacement of 
components if critical resources have failed. 

FRC Configuration 

The previously described basic configuration is the lowest cost alternative, but it does not ensure that 
all errors are detected. For applications that require that all errors are detected, a self-healing system 
can be designed using Functional Redundancy Checking (FRC). Adding a second set of checker 
components to each module improves the error detection capabilities of the system, providing "high- 
confidence" computing. No single hardware failure will go undetected and corrupt the results of a 
critical computation. FRC ensures that any error is caught before it can propagate to another module 
in the system. FRC alone does not provide automatic hardware recovery as in a Quad Modular 
Redundant system, but it does detect and contain errors to ensure that the system does not become 
corrupted. It is then the responsibility of system software to implement a "self-healing" strategy 
where the faulty resource is disabled and the system reinitialized. 



10-9 



iny 



FUNDAMENTAL CONCEPTS OF FAULT HANDLING 







PRIMARY 



SHADOW 



mm 



L-BUS 



L-BUS 



m 



L-BUS 



L-BUs 



,t t 1 1, 

~ AP-BUS ^ 

MASTER CHECKER 



QMR 

PROCESSOR 

MODULE 



m 



m 



L-BUS 



L-BUS 



MZHZH 



FRC 

PROCESSOR 

MODULE 



AP-BUS 



m 



L-BUS 

U|T] 

± 



SINGLE 

PROCESSOR 

MODULE 



AP-BUS 



HARDWARE 
RECOVERY 



HARDWARE 
SELF-CHECKING 



SOFTWARE 

PROGRAMMABLE 

RECONFIGURATION 



LEGEND 

C = M80960MC CPU 
B = M82965 BXU 



INCREASING 
FAULT HANDLING 
PERFORMANCE 



271 079-1 57 B 



Figure 10-5: Fault-Tolerant Alternatives 

Figure 1 0-7 shows a system capable of a comprehensive deferred maintenance. The system uses all 
of the available detection mechanisms, and has more than one of every resource in the system. When 
an error occurs, it will be detected and isolated by the hardware. Software can then rapidly 
reconfigure the system around the faulty module and continue operation. 
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QMR Configration 

In a continuous operation configuration, a redundant resource is used to replace the failed unit when 
a permanent error occurs. This switch is done on a BXU-to-BXU basis; there is no centralized 
element that controls the switch. Each BXU knows which module or AP-bus it is backing up 
(shadowing). If the error report identifies its partner as the faulty unit, then it becomes active and 
takes over operation for the faulty unit. After the resource switch is complete, all of the outstanding 
accesses are retried. This allows operation to resume at a point before the failure could corrupt data. 
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Figure 10-6: Example of a Basic Fault-Tolerant Design 

These reconfiguration and recovery actions are performed by the hardware without any software 
intervention. After recovery is complete, system software can make policy decisions regarding the 
optimum system configuration, given the resources remaining in the system. These policy decisions 
are carried out while normal system operation continues. 
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Figure 10-7: Self-Healing Multiprocessor Configuration 
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Figure 10-8 shows an example of a small continuous operation QMR system. This system is 
configured so that every resource in the system has a backup. The system can continue operation in 
the presence of any single failure. All of the detection and recovery mechanisms are enabled. 
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Figure 10-8: QMR Configuration 

The software configurability of a BXU system allows a system to use a combination of the above 
strategies. In a typical naval avionics application, software can configure a system as a full QMR 
system for critical applications, such as an automatic carrier launch system, and then switch to an 
FRC-only system during flight. The software could subsequently reconfigure the system back to a 
full QMR system for an automated carrier landing system. This reconfiguration doubles the system 
throughput (twice as many processors are working in parallel) without making any hardware 
changes. 

LATENT FAULTS 

Earlier sections of this chapter discussed how hardware faults are detected and handled by the 
interconnect architecture. The interconnect system can be used to detect another class of faults 
known as latent faults. Latent faults can exist in a part of the system which is normally dormant. Under 
normal operation of the system, it would take a long time before the fault reached an interface where 
it would be detected by the other mechanisms. The 80960 interconnect system can be used to exercise 
the normally dormant parts of the system to uncover any latent faults. Table 10-1 summarizes some 
of the areas in which latent faults may exist and the methods used to uncover the faults. 
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Table 10-1: Exercising Latent Faults 



Dormant Area 


Exercise 


Detection Mechanisms 


Software* periodically forces error conditions into the detection 
mechanisms. 


Reporting Mechanisms 


Software* periodically initiates and observes error reports. 


Recovery Mechanisms 


Software* periodically invokes recovery operations. 



NOTE: 

* BXUs use special commands to exercise dormant areas. 

SCOPE OF 80960 FAULT-TOLERANT SYSTEM DESIGN 

The interconnect architecture and the VLSI components provide a stable base for developing fault- 
tolerant 80960 systems. The interconnect components (BXU's) address the issues concerning fault 
tolerance, which are encountered when constructing the 80960 central system. 

A number of system-wide issues remain the responsibility of the 80960 system designer. These 
issues include the following: 

A fault-tolerant asynchronous I/O system 
Fault-tolerant power supplies and distribution method 
A fault-tolerant method for clock generation and distribution 
The electrical and physical provisions for on-line repair 
Environmental conditions, such as system cooling 

SUMMARY 

Faults can occur at various levels within the system. Handling the fault at the lowest possible level 
and not allowing the fault to propagate to a higher level is essential in fault-tolerant design. Higher 
levels have more complex environments, which make recovery a complicated and slow task. As 
failure modes increase in complexity, the interaction between subsystems grows, and the original 
source of the failure becomes more ambiguous. 

The 80960 fault handling approach minimizes the hardware errors which can be reflected to higher 
levels. This is accomplished by duplicating VLSI components, partitioning the system into a set of 
confinement areas, and performing all communication over redundant buses. 

The 80960 AP-Bus interconnect architecture provides a standard VLSI method for constructing 
multiple processor VLSI computer systems. The 80960 interconnect architecture is implemented by 
the 82965 BXU. Together with the 80960MC processor, these components permit the construction 
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of modular, extensible, multiprocessor computer systems. The components are designed to support 
the construction of fully fault-tolerant 80960 systems. 

The fault-tolerant mechanisms are designed to provide a flexible and complete solution to the 
problems of fault-tolerant hardware. For basic systems (those without FRC checkers for error 
detection or QMR for recovery), a user may decide to use only a few detection mechanisms and 
provide recovery only for transient errors. To reduce maintenance costs and increase system 
availability, a system may use all of the detection mechanisms (i.e., may add checker components), 
but may not add any extra recovery capability (i.e., may not configure self-checking modules into 
a fault-tolerant QMR module). Continuous operation is available to the user who adds the extra 
recovery capabilities. 
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CHAPTER 11 
CONFINEMENT AREAS/DETECTION MECHANISMS 

The first step toward fault-tolerant design is to detect and isolate the error. This chapter describes 
the details of the 80960 architecture confinement areas and the associated detection mechanisms. 
Specifically, this chapter explains the following items: 

AP-Bus confinement areas 

• Module confinement areas 

• Functional Redundancy Checking (FRC) 

CONFINEMENT AREAS 

A confinement area limits error propagation and localizes the faulty area for subsequent recovery and 
repair. Confinement areas are defined as units of the system which have a limited number of tightly 
controlled interfaces. Detection mechanisms are placed at every interface to ensure that no 
inconsistent data can leave an area and corrupt other confinement areas. As data flows from one area 
to another, the operation of the confinement area is rigorously checked and correct operation is 
guaranteed before it is passed on to the next confinement area. 

Figure 11-1 illustrates the different types of confinement areas. The system bus confinement area 
consists of the AP-bus and part of the BXU. The module confinement area consists of the BXU and 
all of the components attached to its L-bus except for an asynchronous I/O port. The interface to an 
asynchronous I/O system is considered in Chapter 15. 
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Figure 11-1: Fault Tolerance Confinement Areas 
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THE AP-BUS CONFINEMENT AREA f 

The AP-bus confinement area consists of the AP-bus and some of the interface logic in the BXU. 
Every BXU attached to AP-bus checks the information that flows from the AP-bus to the module. 
No infonrMibri can leave the AP-bus cbnfinerrient are^ and enter a module without first being 
checked by the BXU. To Accomplish this, the BXU uses the following error detection mechanisms: 



1. Two interlaced parity bits covering the AD 31 -AD and SPEC 5 -SPEC signals 

2. Signal duplication for ARB 3 -ARB , and RPYDEF 

3. Bus time-outs 

Parity 



Two interlaced parity bits (CHKj and CHK ) are provided to check the transmission of the address, 
data, and specifica tion in formation; Interlaced parity associates adjac ent sig n als wit h different parity 
bits. For example; CHK is even parity across the eve n AD^ -AD n and SPEC 5 -SPEC pins; and CHKj 
is even parity across the odd AD 31 -AD 6 and SPEC 5 -SPEC pins. This alignment allows the parity 
detection mechanism to detect the most frequent double failure eases (shorts between adjacent lines), 
as well as all single bit failures. The parity bits are always valid, even when the AP-bus is idle. 



The value of CHKj and CHK signals are compared with the values calculated internally by the BXU. 
For proper timing, the receiving BXU waits 0.5 bus cycles after receiving the data before issuing a 
parity error. ■ ■'■'•'":■' 

Signal Duplication (optional) 



The ARB 3 ,-ARB , and RPYDEF signals are asserted independently by multiple bus agents. 
Consequently, parity bits cannot be used to check for a correct tr^ 

Detection of errors pn these lines can be performed by duplicating each set of lines, one set for 
masters, and the other for checkers of an FRC pair (see the following "Functional Redundancy 
Checking"1sectionf. If there is a disagreement betw een the two pins for each sej of signals on each 
AP-bus, it is detected by FRC disagreement on the BOUT lin6 associated with that bus. 

Bus Time-Outs 

All the BXUs contain a timer that is used to detebtbus accesses to non-existent or railed components; 
During ribfmal df*erMbri; the tirfter is active whenever the bus plpelifie is not empty; The timer on 
the BXU starts when its request enters slot one of the pipeline queue (see the "Reply Ordering" 
section in Chapter 7). When this BXU receives a reply or a reply deferral packet, the timer is reset. 

The timer is nominally 64 bus clock cycles long. A time-but occurs if the bus is idle for more than 
64 bus cycles while there is an outstanding request. When a time-out occurs, the BXU that originally 
sent the request issues a BAD-ACCESS reply code in the SPEC field reply packet (to the originating 
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CPU) to clear the request from the bus. This action prevents the 80960MC processor and the AP- 
bus from suspending activity because a destination does not exist. 

An access may be allowed to take longer than 64 bus cycles, but the serving BXU must assert the 
RPYDEF signal. The timer cannot be disabled. 

MODULE CONFINEMENT AREA 

The module confinement area contains the 80960MC processor, the associated BXUs (except for the 
internal logic used by the AP-bus confinement area), the support logic in the module, the RAM array, 
the memory controller, etc. The module confinement area includes all the components that reside on 
the L-bus except for any asynchronous I/O system. The AP-bus is the only interface to the module 
confinement area. No information (control, address, or data) can leave the module confinement area 
without first being checked by one of the BXUs in the module. The interface to an asynchronous 
I/O system is considered in Chapter 15. 

The error detection mechanisms for the module confinement area are similar to the AP-Bus 
confinement checks (interlaced parity, signal duplication, bus time-outs), with the addition of 
Functional Redundancy Checking (FRC). 

FUNCTIONAL REDUNDANCY CHECKING 

Functional Redundancy Checking consists of a clock-by-clock comparison of the AP-bus pins of two 
BXUs. Each pair of BXUs act as a single logical unit with one operating as a master and the other 
as checker. These two bus agents run in lockstep, with the checker ensuring that the master is 
operating correctly. On every clock cycle, the checker compares its internal bus data to the data it 
actually observes on the AP-bus, which is driven by its master. If a disagreement is detected, then 
an error has occurred in either the master or checker. Also, when operating as an FRC pair, each BXU 
checks receipt of information from its shared local bus and transmission over the AP-Bus. 

In systems that use FRC, the quality of error detection on the AP-bus is increased. FRC provides 
detection of errors that ma y occu r in the int erface to th e AP-bus and BXU. In addition, FRC allows 
detection of failures in the ARB 3 -ARB or RPYDEF signals for a dual arbitration network. 

Physical Connection 

For FRC to exist, the two BXU's must be tightly coupled to form a master-checker pair. Figure 
1 1 -2 shows the interconnection of the FRC pai r. Each BXU of an FRC pair must attac h to the same 
AP-Bus, and the bi-directional Module check (MODCHK) and Bus Output Control (BOUT) lines 
must be connected together in a point-to-point manner between the FRC pair (ie. not part of the AP- 
bus). 



The MODCHK line is asserted by t he BXU to i nform its FRC twin that it has received a request from 
the L-bus for access to the AP-bus (MODCHK does not check whether or not the signals are correct). 
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The BXUs detect an FRC error when one BXU asserts MODCHK while the other does not. Both 
the master and the checker hav e the capability to detect a module checking (FRC) error. If a BXU 
is not asserting the MODCHK line and it senses that it is asserted (by its twin or a hardware fault), 
it will generate an "Unsafe Confinement Area" FRC Error report. This type of error indicates that 
the FRC pair has lost synchronization and is considered a permanent failure, since it cannot be 
determined which of the components is at fault when a disagreement occurs. 
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Figure 11-2: Functional Redundancy Checking 



If the request for AP-Bus access has been successfully received by both BXU's without a MODCHK 
generated FRC error, the arbitration process for AP-Bus access begins. Both the master and checker 
BXU's arbitrate for the AP-Bus. After the master BXU has gained access to the AP-Bus, it drives 
the bus through its AP-Bus output buffers. 



The Bus Output Control (BOUT) line is asserte d by the BXU to inform its twin that it is driving the 
AP-Bus. Although both BXU's assert BOUT, only the master actually drives the AP-Bus. The 
checker's output d rivers a re internally disabled, as it only checks the data on the AP-Bus; it never 
drives the bus. The BOUT line is used to detect arbitration errors by checking whether or not the BXU 
drives the AP-bus (BOUT does not check whether or not the signals are correct). The paired BXUs 
det ect an F RC error when one BXU asserts BOUT while the other does not. If a BXU is not asserting 
the BOUT line and it senses that it is asserted (by its twin o r a hard ware fault), it will generate a "Bus 
Arbitration" FRC Error report. Errors detected by the BOUT line may be either permanent or 
transient since they can be induced by a hardware problem or noise on the AP-Bus. 



SPEC,-SPEC n , 



A bit-for-bit comparison (exclusive or) is performed by the FRC pair on the 
AD 31 -AD lines, to check whether on not these signals are correct. FRC on the SPEC 5 -SPEC and 
AD 31 -AD lines provides detection for any data manipulation, data translation, or sequencing errors. 
The master drives the lines, and the checker verifies the validity of the data. The master also performs 
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a self check to verify the state reflected on it's output drivers matches the data it is driving onto the 
AP-Bus. Either the master or the checker can detect this type of FRC error, and will generate a 
"Component" FRC Error report. Functional Redundancy Checking can be disabled by clearing the 
Error-Reporting-Enable bit in the FT1 register, or by splitting the FRC pair (see Appendix A for the 
description of the FT1 register). 



Func tional Redundan cy Checking is not performed on th e BERL r BER L n , CHKj-CHK^, ARB 3 - 
ARB , and RPYDEF lines. FRC does not apply to the BERL,-BERL lines beca u se the error 
reporting protocol checks these lines. Similarly, FRC does not apply to the CHKj-CHK lines 
because their operation is fully checked by the parity computation. 



The ARB 3 -ARB and RPYDEF pins can be configured in two ways: in a dua l ar bitration ne twork, 
or a single arbitration network. In dual arbitration network systems, the ARB and RPYDEF signals 
are duplicated; one set connects to the masters, the other connects to the checkers, as shown in Figure 
11-3. The duplication of these signals forms two parallel networks so that an arbitration error can 
be detected as an FRC error on the BOUT line. This configuration allows the detection and 
containment of arbitration errors before they can corrupt the system bus, but does not allow the FRC 
pair to be split. 
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Figure 11-3: Dual Arbitration Network for Each AP-Bus 
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In systems using a single arbitration network on each AP-bus, the ARB 3 -ARB and RPYDEF pins 
are connected to gether , as sho wn i n Figure 1 1 -4. This configuration allows the FRC pair to be split, 
but errors on the ARB 3 -ARB and RPYDEF may not be detected before the master drives the AP- 
Bus. 
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Figure 11^4: Single Arbitration Network for Each AP-Bus 

FRC Operation and Setup 

During system operation, either the master always drives the AP-bus and the checker monitors the 
AP-bus, or the master and checker alternate driving the bus after each bus clock cycle. This option 
is controlled by the Toggle-Master I Checker bit in the FRC register (see Appendix A for the 
description of the FRC register). The Master bit in the FRC register must be set in one of the pair 
and cleared in the other. The Separated-MIC bit in the FRC -Splitting-Control register in each BXU 
must be cleared (see Appendix A for the description of the FRC -Splitting -Control register). 

EXAMPLE OF OPERATION USING CONFINEMENT AREAS 

An example of processor-memory operation may help to clarify the operation of the confinement 
areas. This example is shown graphically in Figures 1 l-5a through 1 l-5d, with each logical module 
representing an FRC pair. Assume that an 80960MC processor makes a read request to a memory 
location. This request is mapped to a BXU on the addressed AP-bus (FIG. 11 -5a). As the information 
flows onto the AP-bus, it is checked by the BXU. If any failure has occurred in the CPU confinement 
area (80960MC processor, L-bus, BXU, etc.), it is detected at this time. If no error is detected, the 
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information passes through the BXU and flows across the AP-bus to the addressed memory module. 
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Figure 11-5: Confinement Area Operation 
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Before the information is accepted by the memory module, that module's BXU checks it for 
accuracy. If a failure is detected, it is confined to the AP-Bus because the information was valid when 
it left the CPU module confinement area (FIG. 1 l-5b). If no error is detected, the serving (Memory 
Module) BXU performs the requested memory operation and returns data to the AP-bus. As data 
flows onto the bus, the serving BXU checks it for accuracy (FIG. 1 l-5c). As the data flows into the 
CPU module from the AP bus, the requesting BXU checks it for accuracy before being used by the 
CPU module (FIG. 1 l-5d). If an error was detected at this point it would be confined to the AP-Bus, 
and not permitted to corrupt the Module. 

The confinement area interfaces provide tight error control and isolate the failure to one of the 
building blocks present in the system. 

SUMMARY 

This chapter discussed confinement areas and fault detection methodology in an 80960 system 
utilizing the M82965 BXU component. 

The 80960 fault- tolerant system uses two types of confinement areas: the AP-bus confinement and 
module confinement areas. Faults are prevented from crossing confinement area boundaries and 
corrupting other areas. Errors are contained within these confinement areas by using detection 
mechanisms such as parity, signal duplication, bus time-outs, and FRC. Therefore, faults are 
contained at the lowest hardware level for subsequent reporting and recovery mechanisms. The 
80960 Fault Tolerant error reporting system is discussed in detail in Chapter 12 and error recovery 
is presented in Chapter 13. 
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CHAPTER 12 
ERROR REPORTING 

Error reporting is the backbone of fault isolation and recovery. The error reporting system is designed 
so that, independent of the errors in the system, each BXU receives the same error report. When a 
BXU detects a fault, it broadcasts a serial error message to all other BXUs in the system. This error 
report message identifies the fault confinement area and the type and location of the component 
reporting the error. Sending this error report accomplishes two objectives: it informs the rest of the 
system that an error has occurred to prevent other confinement areas from using the inconsistent data, 
and it provides the necessary information for system recovery. Each BXU records this error message 
in the Error-Log register and proceeds independently with the appropriate recovery procedure. 

This chapter describes the details of the error reporting procedure, specifically covering the 
following topics: 

• The topology of the error reporting network 

• The error reporting protocol including a description of how error reports are propagated 
throughout the system 

• The error message format 

• The types of error reports that are broadcast 

• Error recording 

Error reporting diagnostics 

TOPOLOGY OF THE ERROR REPORTING NETWORK 

The serial error reporting network f ollows t he sam e matrix topology as the AP-bus and th e L-bus . 
Two A P-bus error reporting lines (BERL^BERLq) and two L-bus error reporting lines (LERLj- 
LERL ) are used to report the s erial er ror messages. The BERL lines are connected to the other BXUs 
on the AP-bus. Similarly, the LERL lines are connected to other BXUs on the L-bus. An identical 
serial error report is sent over each of the pair of lines associated with a bus. 

The two error reporting lines that accompany each bus ensure that the error reporting network 
functions correctly independent of a failure/If either line should fail, the second line will continue 
to provide a correct error report. This level of redundancy ensures that the error report message 
reaches all compo nents of the bus to prevent corruption of the other confinement areas. In addition, 
one of the LERL lines may be connected to one of the active low interru pt pins of tHe 80960MC 
Processor, so that the processor is notified of ah error when it occurs. (The LERL line must be tied 
through a flip-flop to prevent every active low bit in the error message from causing an interrupt.) 

Because the error reporting network is associated with multiple AP-buses, another level of 
redundancy is reached. Failures in the error reporting network are either inside the AP-bus 
confinement area or the module confinement area. Thus, the replication of A^-buses and BXUs (a 
totally independent BERL network is implemented on each AP-Bus) provides another duplication 
of the reporting lines. As long as the system is connected by functioning AP-buses, it also has a fully 
connected and functioning error reporting network. 
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ERROR REPORTING PROTOCOL 

The error reporting protocol ensures that all components receive the error message in a timely 
manner. The reporting period lasts 256 AP-bus clock cycles and consists of two phases, as shown 
ih Figure 12-1 ; The BXUs send the same error message twice during each phase. The error message 
consists of duplicate error reports. 
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Figure 12-1: Error Reporting Period 

If multiple error messages are broadcast, the protocol ensures that only one message is received by 
the components in the system. The description of the system- wide error reporting protocol is divided 
into the following three segments: 



BERL f BERL timing 
Phase one of error reporting 
Phase two of error reporting 



BERL^BERLo Timing 

The BXUs send serial error messages at half the speed of the AP and L buses. Thus one bit is 
transmitted every two bus cycles (four CLK2 cycles). The error message begins with the Start bit 
followed by an additional 49 bits. This section provides a detailed description of the timing for the 
Start bit and the BXUs response to the start bit. The "Error Message" section of this chapter describes 
the contents of the error message. 

The start of an error message indicates that all of the currently outstanding bus transactions should 
be retried because they may contain incorrect information. Because the AP-bus cycle is operating 
twice as fast as the error reporti ng cycl e , the Sta rf bit, which can occur on any AP-bus cycle, may 
last for only half of the normal BERI^-BERI^ bit time. The Start bit is only sent once; it is not 
repeated when the error message is sent again. 

Two modes are available for controlling the timing of the BXU's reaction to errors, as shown in 
Figure 12-2. If the BERL-Wait bit in the FT1 register(see Appendix A for the description of the FTl 
register) is set ^ (palled W ait-For-BERL timing), then the data received from the AP-Bus is held for 
1 .5 bus cycles by the BXU before it places the data on the L-bus. If the bit is not set (called No-Wait- 
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For-BERL timing), then the incoming data will be held for only 0.5 bus cycles by the BXU before 
it places it on the L-bus. 
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Figure 1 2-2: Timing for the Start of an Error Report 

If the No-Wait-For-BERL timing mode is selected, the receiving BXU places the data on the L-Bus, 
0.5 cycles after the data was valid on the AP-Bus, provided the following conditions exits: 



• The BERLj-BERI^ lines are not asserted. 

• No parity error is detected. 

If a parity error is detected internally by the receiving BXU or if either of the BERL lines were 
asserted, then this cycle is discarded and the request is retried. This mode allows correct recovery 
from parity errors detected by the B XU and from ECC errors detected by a memory control unit as 
reported by the BXU on the BERL lines. FRC errors and parity errors detected by other BXUs are 
not reported soon enough to stop internal acceptance of data. No-Wait-For BERL provides high 
performance in normal operation, but sacrifices some of the recovery capabilities that are available 
with the longer hold time. 



If either a parity error is detected or either of the BERL lines were asserted, then this cycle is 
discarded, and the request is retried. This mode of operation allows correct recovery from parity 
errors detected by the BXU. FRC errors and parity errors detected by other BXUs are not reported 
soon enough to stop the internal acceptance of data. This provides higher performance in normal 
operation, but sacrifices some of the recovery capabilities that are available with the longer holdtime. 



If the Wait-for-BERL timing mode is s elected , then the BERL lines are checked 1.5 bus cycles after 
receiving data/address. If either of the BERL lines were asserted, then this cycle is discarded (data 
not placed on the L-Bus) , and the request is retried. The reaction is always the same, because the extra 
cycle allows all decisions to be based solely on the state of the BERL lines, rather than using the 
internal parity information. This mode allows correct recovery from any type of error detected in the 
system. 
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Figure 12-3: Error Report Propagation for Phase One 
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Phase One of the Error Reporting Sequence 

During phase one of the error reporting cycle, the BXUs communicate the error message to each other 
throughout the system. The BXU detecting the error sends a serial error message along both BERL 
lines (see Figure 12-3a). All BXUs terminate their AP-bus operations upon receiving the Start bit of 
an error report message. This propagates the error message to all BXU's residing on the same AP- 
Bus as the BXU that detected the error. In the case of multiple errors being detected, the BXU 
generate s only the highest priority error report on the bus. Those BXUs that receive an error message 
on their BERL lines first are called originating BXUs. 



Four AP-bus clock cycles after the serial mess age sta rted on the BERL lines, the originating BXUs 
in Processor mode issue an error report on bot h LERL lines on their L-buses (See Figure 1 2.3b). The 
BXUs that receive an error message on their LERL lines f irst are called propagating BXUs. The 
propagating BXUs send the error message on their BERL lines after a delay of four AP-bus clock 
cycles (See Figure 12-3c). 



Figure 12-4 shows the timing of phase one and its relationship to phase two (pha se two i s described 
in the next section). The propagating BXUs issue the error message on their BERL lines eight 
AP-bus cycles after the originating BXU s durin g phase one. Whereas, the originating and propagat- 
ing BXUs issue the error report on their BERL lines at the same time during phase two. 
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Figure 12-4: The Timing of the Error Report Propagation 

The above scenario assumes that the BXU receives an error report from an active AP-bus, i.e., the 
Inactive bit of the FT I register was set to a value of zero. If the BXU re ceives an er ror rep ort from 
an inactive bus, it does not propagate the error report from the BERL lines to the LERL lines (it 
ignores the error report). In the event that the BXUs must attach to an inactive AP-bus, the BXUs 
transmit an Attach-Bus error report over the error reporting network (see the appendix for a 
description of the "Attach-Bus" command) to clear the Inactive bit in the FT1 register. 

If multiple error reports occur on the AP-bus, the BXU selects the first error rep ort that it receives 
during an error reporting cycle. If a BXU rece ives an error report on its BERL lines, and 
simultaneously receives another error report on its LERL lines, it does not propagate either error 
report until phase two of the reporting sequence. (See the next section for the description of phase 
two.) 
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Several BXUs can react to a single error condition on either the BERL or the LERL lines and can 
simultaneously generate error reports. In this case, the BXU propagates only the highest priority 
error report, which is based upon the contents of the error report message (see the "Error Message" 
section of this chapter). 
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Figure 12-5: Simultaneous Error Reporting Example 

For example, consider the configuration show n in F igure 12-5. Upon detection of an error, the 
BXU's generate serial error reports over their BERL lines. The error report is comprised of the 
ERROR TYPE followed by the CONFINEMENT IDENTIFICATION (and several other bits not 
relevant in this example). The most significant bit (MSB) of each field is transmitted first, and the 
least significant bit (LSB) is sent last. Assume that BXU B generates an error report with an error 
type value of 1 1001 and a location identification (Confinement-ID field) of 0000001 1. Assume that 
BXU C simultaneously generates an the same error report (an error type value of 1 1001), and it's 
Confineme nt-ID of 00000 1 0. BXU B and BXU C simultaneously issue the error message on their 
respective BERLj-BERL^ lines. BXU A receives the error message from BXU B and begins to 
propagate th is error message on its LERL lines at the same time BXU C propagates its error message 
on the same LERL lines. 

Each BXU continues to propagate the error messag e until a difference is detected. At this point, the 
BXU with the higher priority (the BXU with its BERL bit asserted) continues to propagate the 
message, while the other BXU begins to receive the message. In this example, both BXU A and BXU 
C propagate the error type and the first seven bits of the Confinement-ID because these values are 
identical. However, during the transmission of the eighth bit (the Confinement field LSB), BXU C 
continues to propagate its message, while BXU A begins to receive the error message. 

At the end of phase one, BXU A and BXU C store the same error message in their Error-Log register, 
while BXU B stores the lower priority error message in its Error-Log register (see Appendix A for 
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the description of the E rror-Log register). In phase two, BXU A and BXU C generate the same error 
report on their BERL lines. BXU B does not generate an error report in phase two because it is in a 
module which is not fully connected to al l AP-b uses. Four clock cycles later both BXU A and BXU 
C will generate the error report on their LERL lines and upon completion, all modules will have 
received the correct error message. Thus the error message conflict is resolved during phase two 
when the error message is broadcast again. This mechanism eliminates the lower priority error 
reports and makes sure that all components in the system receive the same error report message. 



If a rac e (simultaneous reception) condition exist s betwe en error reports on the BERL lines and the 
LERL lines, the BXU ignores the message on the BERL lines and uses the information o n the L ERL 
lines, as this error report must have occurred prior to the error repo rt just r eceived on the BERL lines 
(the BXU does not propagate the error message received on the BERL lines). 

For example, consider the configuration in Figure 12-5 again. For this case, assume that B XU C 
reports an e rror th at occurred on AP-bus,. BXU C issues an error message first on the BERL lines, 
then on the LERL lines fo ur AP-bus clock cycles later. At the same time th at BXU C is issuing the 
error report on the LERL lines, BXU B begins iss uing an error report on its BERL lines. Therefore, 
BXU A recei ves error messages on its BERL and LERL lines at the same time. The error message 
on the BERL lines is ignored because it had t o have occurred after the error being r eported on the 
LERL lines. The error received on the LERL lines is then propagated on the BERL lines by BXU 
A. Consequently, two error reports exist at the same time on the BERL lines associated with AP-bus Q . 

During pha se one, BXU A and BXU C store the original error report (originated by C and propagated 
to A on the LERL lines) in their Error-Log registers and BXU B stores the second error report in its 
Error-Log register. 



Du ring ph ase two, BXU A and BXU C propagate the original er ror rep ort over BERL lines, followed 
by LERL lines. BXU B receives the correct error report on its BERL lines and stores this report in 
its Error-Log register. Therefore the error report from BXU C is acted upon and stored in all Error- 
Log registers. 

Phase Two of the Error Reporting Sequence 

Phase two resolves any ambiguity that exists if multiple error messages were issued during phase one. 
If a BXU received at least one valid report message and the BXU is in Processor mode, then the 
highest priority message or first error message receive d durin g phase one is repeated. As shown in 
Figure 12-6, the error message is first repeated on the BERL lines, then the message is sent on the 
LERL lines. If a BXU did not receive any valid error messages, then it does not broadcast any error 
messages during phase two. 

The BXUs in Memory mode do not send error reports during phase two because they are not required 
to be in a module that is fully connected to all AP-buses. Thus, these BXUs may have an incorrect 
view of the resolution of any conflicts between multiple error reports. 

All BXUs in a subsystem receiving a valid error report issue their phase two report at the same time. 
Phase two begins 30 cycles after the end of the original report. 
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At the end of the error reporting cycle, the BXU copies the previous contents of the Error-Log register 
into the internal Error-Record register and places the last error message received in the Error-Log 
register (see Appendix A for the description of the Error-Record register). If the BXU does not 
receive any valid error messages, then the Error-Log register contains no useful information, and the 
Invalid bit in the Error-Log register is set (this is not to be confused with the Invalid bit in the Error 
Report). The BXU operates its recovery algorithm on the final error message it receives, whether it 
was generated on its AP-bus or propagated from another bus. 
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1. THICK LINE REPRESENTS THE ERROR PROPAGATION PATH. 
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Figure 12-6: Error Report Propagation for Phase Two 

ERROR MESSAGE FORMAT 

Figure 12-7 illustrates the format and sequence for the 50-bit serial error message, which consists of 
aJStart bit, Null bits, an Error Type field, a Location-ID field (comprised of the Confinement-ID and 
Source-ID sub-fields), an Invalid bit, and parity bits. The sequence of events for the error message 
is listed below: 
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1 . Send a Start bit followed by two Null bits. 

2. Broadcast the 22-bit error report twice to protect against burst errors. 

3. After two Null bits, an Invalid bit if the error report has an error that did not occur in this 
confinement area. 
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Figure 12-7: Error Message Sequencing 

A description of each bit or field is provided in the following list. Note that the Location-ID field 
is further divided into the Confinement-ID and the Source-ID fields. 



Start 



This bit is used for two purposes: to indicate the start of an error 
message and to stop data processing. The Start bit (asserted low) 
marks the beginning of the reporting phase of a fault handling 
cycle. 



Null 



These two bits have a "don't care" value. This time period is used 
by the fault-tolerant logic of the BXU to process the error informa- 
tion and set up the propagation paths for the error report. 



Error-Type 



Type-Parity 



Confinement-ID 



This five-bit field specifies the type of error that was detected. The 
field is fully encoded, providing up to 32 prioritized error types 
(only 15 are currently used): error type 3 1 is the highest priority and 
error type is the lowest. The different error types are described 
in the "Types of Error Reports" section of this chapter. 

This parity bit provides odd parity over the Err or -Type field. It 
does not cover any other bits in the message. This parity bit is 
generated internally by the BXU generating the error message. 

This eight-bit field uniquely identifies a single confinement area 
where the error has occurred. The value for this field is the same 
as Confinement-ID field from either the Module-Err or -ID register 
(for the module confinement area) or the Bus-Error-ID register 
(for the AP-bus confinement area) (see Appendix A for the 
description of the Module-Err or -ID and Bus-Error-ID registers). 
The BXU selects the correct Confinement-ID field to broadcast. 



12-9 



iny 



ERROR REPORTING 



Source-ID This seven-bit field identifies the component reporting the error. 

This field is used by the diagnostic software to help isolate bus 
problems or provide recovery in self-healing systems. The value 
for this field is the same as the Source-ID field of either the Module- 
Err or -ID or Bus-Error-ID register when the BXU reports an error. 
The BXU selects the correct Source-ID field to broadcast. 

ID-Parity This parity bit provides even parity the ID -Parity bit from either 

the Module-Error-ID or the Bus-Error-ID register, depending on 
the one used for the error report identification. 

Invalid When asserted, the Invalid bit indicates that an error message 

contains an error that did not occur in this confinement area. This 
is necessary because the BXU may propagate an error report that 
contains errors. 

TYPES OF ERROR REPORTS 

The five-bit Error-Type field in the error report specifies what type of error has occurred. This sec- 
tion describes the details of the 15 types of errors that can occur. 

Error Priorities 

The following list consists of the errors reported by the BXU in priority order, the highest priority 
is listed first. The number in parentheses is the priority number (in decimal notation). Each error 
type identifies a faulty confinement area, as denoted by the Location-ID field in the error message. 

The Location-ID field reported in the error message is from either the Module-Error ID register or 
the Bus-Error-ID register depending upon the type of error. The Location-ID field assignment is 
shown in brackets at the end of each error type. All error types use the Module-Error-ID register for 
the Location-ID field if the Bus-Switch-Disable bit in the AP-Control register is set (see Appendix 
A for the description of the AP-Control register). This process provides the best chance for recovery 
in a single bus system. By taking the module out of operation, there is some probability that the rest 
of the system is able to continue. If the AP-bus were removed, the system would stop operating. 

Unsafe-Confinement-Area (31) 

The BXU issues this report when a confinement area has an error that makes a retry operation 
dangerous. This is the highest priority error report and immediately causes a permanent error. The 
BXU generates this error report in the following cases. 



The current error report on the BERL lines has an Error-Reporting-Error type associated with 
it and the previous report contains an Error-Reporting-Error type. 
[Location-ID = Bus-error-ID] 



The current error report on the LERL lines has an Error-Reporting-Error type associated with 
it and the previous report contains an Error-Reporting-Error type. (If the BXU that originates 
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the error report detects an error on both error reporting lines, then the Module-Error-ID is used 
as the Location-ID field.) 
[Location-ID = Module-Error-ID] 

The External Error (ERR) input signal is asserted on a BXU in Memory mode. This type of error 
may be used by external circuits to notify the system of an external fault. 
[Location-ID = Module-Error-ID] 

• An FRC error is detected on the MODCHK pin. This is an unsafe error because the internal signal 
to initiate arbitration does not occur during the retry sequence. Hence, the module failure would 
be incorrectly reported as a bus failure. 
[Location-IC = Module-Error-ID] 

Primary-Catastrophe (30) 

The BXU generates this error report when it receives a Primary-Catastrophe command (see 
Appendix A for the description of the Primary-Catastrophe command). Software should issue this 
command whenever a condition external to the BXU indicates all primary units are about to fail, such 
as failure of the power supply for the primary units. This error report immediately causes a permanent 
error. The contents of the Module-Error-ID register are used for the Location-ID field. 

Shadow-Catastrophe (29) 

This error report is similar to the Primary-Catastrophe error report, except that it indicates that all 
shadow units are about to fail. The BXU generates this error report when it receives a Shadow- 
Catastrophe command (see Appendix A for the description of the Shadow-Catastrophe command). 
The contents of the Module-Error-ID register are used for the Location-ID field. This type of error 
report is treated in the same manner by hardware and software as the Primary-Catastrophe error 
report except that it pertains to shadow modules. 

Error-Reporting-Error (28) 

This report indicates that a component has detected a failure in the error reporting lines. The duplicate 
error reporting lines on each AP-bus allow the BXUs to issue this type of error report. The contents 
of the Bus-Error-ID register are used for the Location-ID field if the error occurs on the AP-bus, or 
the contents of the Module-Error-ID register if it oc curs o n the L- bus. The fa ilure is cause d by the 
following conditions: bad parity on either the BERLj or the BERL line (or the LERLj or the LERL 
line) in either copy of the error report, or a mismatch between the two copies of the error message 
within a phase of error reporting. 

Bus-Arbitration (27) 



This error report is issued when the BXU detects a bus arbitration error (by using FRC on BOUT). 
The contents of the Bus-Error-ID register are used for the Location-ID field. 
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Bus-Parity (26) 



This er ror report is issued when the BXU detects a bus parity error on the AD 31 -AD or the SPEC 5 - 
SPEC lines. The contents of the Bus-Error-ID register are used for the Location-ID field. 

Component (25) 



This error report is issued when the BXU detects an FRC error on the AD 31 -AD or SPEC 5 -SPEC 
pins on the AP-bus interface. The contents of the Module -Error-ID register are used for the Location- 
ID field. 

Uncorrectable-Array-Error (23) 

This error report is issued when an uncorrectable error is detected in the mem ory ar ray. The BXU 
is informed of this condition when the external memory controller asserts the ECC and UNC pins. 
The contents of the Module-Err or -ID register are used for the Location-ID field. 

Correctable-ECC (22) 

This error report is issued when a correctable error is detected in the m emory array. The B XU is 
informed of this condition when the ex ternal mem ory co ntrol ler ass erts th e ECC pin, but not the UNC 
pin, and the BXU is not asserting the COR pin. (ECC •notJJNC • not_COR) The contents of the 
Module-Err or-ID Register are used for the Location-ID field. 

COM- Altered (21) 

This error report is issued by the BXU when the COM input is toggled from high-to-low, and the 
COM-Reporting bit is set in the FT1 register. This error report can be used by external hardware 
to notify the system of a change in status of the board. The contents of the Module-Error-ID register 
are used fox Location-ID field. 

Attach-Bus (19) 

This error report is generated by the BXU in response to an Attach-Bus command. The contents of 
the Bus-Error-ID register are used for the Location-ID field (see Appendix A for the description of 
the Attach-Bus command). 

Detach-Bus (18) 

This error report is issued as a result of an Detach-Bus command. The contents of the Bus-Error- 
ID register are used for Location-ID field (see Appendix A for the description of the Detach-Bus 
command). 
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Terminate-Permanent-Error-Window (17) 

This error report is issued as a result of a Terminate-Permanent-Error-Window command (see 
Appendix A for the description of the Terminate-Permanent-Err 'or '-Window command). The 
contents of the Module-Error-ID register are used for the Location-ID field. Receiving this report 
ends the permanent error window period. 

Sync-Refresh (16) 

This error report is issued whenever the Sync-Refresh command is received by the BXU (see 
Appendix A for the description of the Sync-Refresh command). The contents of the Module-Error- 
ID register are used for the Location-ID field. 

All error types use the Module-Error-ID register for the Location-ID field if the Bus-Switch-Disable 
bit in the AP -Control register is set at a value of one (see Appendix A for the description of the AP- 
Control register). This process provides the best chance for recovery in a single bus system. By 
taking the module out of operation, there is some probability that the rest of the system is able to 
continue. If the AP-bus were removed, then the system would stop operating. 

No-Op (0) 

Error type zero is reserved as a no-op because zero is the default value loaded into the Error-Log 
Register at RESET. 

Error Types for Different Detection Mechanisms 

Table 12-1 lists the Error-Type field that is generated by each of the detection mechanisms in the 
BXU. 

It is possible for a single failure to be detected as multiple error types by different units on the bus. 
Error types are prioritized to make sure that the location of the failure is correctly identified. When 
multiple errors are detected (or reported), the error report with the highest priority error is selected 
by the BXU for recovery. 

Using Commands to Generate Error Reports 

Some of the error types are generated as a result of commands to the BXU. Prior to issuing the 
command to the BXU, the Testing-Enable bit in the Test-Detection register must be set (see 
Appendix A for the description of the Test-Detection register). Before setting the Testing-Enable 
bit, there should be a single bit with a value of one in the COM register. When the command is 
executed, the Testing-Enable bit is cleared. If latency is important, then the Testing-Enable bit may 
remain set. In this way, no extra accesses are required at the time of the command. 



12-13 



inter 



ERROR REPORTING 



Table 12-1: Error Types for Detection Mechanisms 



Detection Mechanism 


Error Type 


FRC on AD or SPEC signals 


Component 




Bus-arbitration 


FRC on BOUT signal 




Unsafe-confinement-area 


FRC on MODCHK signal 


AP-bus parity 


Bus-parity 


Error on BERL r BERL or LERL r LERLo 


Error-reporting-error 




Unsafe-confinement-area 


Error on BERL^BERU or LERU-LERLq with error-reporting-error'm 
error-log register having the confinement-ID field of this BXU's 
AP-bus or module, respectively. 


COM input pin 


Corn-altered 


E5Cpin(ECC»UNC«COR) 


Correctable-ECC 


ECC and DNC pin 


Uncorrectable-array-error 


ERR pin 


Unsafe-confinement-area 



ERROR REPORT LOG 

Two registers are used in recording the error report: the Error-Log register contains the current error 
report and the Error-Record register contains the previous error report. When an error report is 
received, the contents of the Error-Log register are copied into the Error-Record register. The Error- 
Log register is accessible to software and represents the primary form of communication between the 
hardware fault-handling mechanisms and the software routines responsible for system management. 
The information contained in the Error-Log register always reflects the latest error report generated 
in the entire system. 

The Error-Log register is used independently by the hardware and software. The hardware recovery 
procedures use the Error-Type and Location-ID fields in determining which recovery actions are 
performed. The hardware reacts immediately to each error report; thus, it never encounters an 
overflow condition. 

To determine if the current error is permanent, the BXU utilizes the Error-Record register. The error 
is treated as permanent if the contents of the Error-Record register match the Error-Log register after 
a retry operation. In other words the same error occurred twice in a row within the Permanent Error 
Window. 
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The BXU provides a software-visible, as well as an external indication of a permanent error in a 
module. Software can read the Error-Log registers to determine whether an error is permanent. The 
BXU asserts the COM pin to i ndicate a permanent error in the hardware system. For details on COM 
pin activation, see the "Serial COM Protocol" section in Chapter 14. 

ERROR REPORTING DIAGNOSTICS 

The internal fault-tolerant logic of the BX U provi des all the standard error checking and reporting 
capabilities that are communicated over the BERL lines. This logic and the error reporting network, 
however, are also susceptible to failures. The BXU takes care of th is situation by making provision 
for diagnostic tests of the fault-tolerant logic and the BERLj-BERI^ network. This section examines 
these cases. 



BERL^BERLo Error Detection 

Because the error reporting system is fundamental to the correct operation of a fault- tolerant system, 
error detection mechanisms are built into each error report network. These mechan isms en sure that 
the error report network operates correctly in the presence of single failures. The BERL lines use 
the following two error detection mechanisms: 

1 . Parity on the error message (Type and Location-ID fields) 

2. Error report duplication (The sequence is repeated twice for a single error report). 

These mechanisms detect an error in the error reporting network, itself. The Invalid bit at the end 
of the error report is used by the BXU to indicate whether the error reporting error originated in this 
confinement area or not. The BXU that originates the error report always clears the Invalid bit. If 
the BXU that propagates the error report detects an error in the error message, it performs one of the 
following actions. 

1 . If the Invalid bit is cleared, the BXU knows that the error originated in the confinement area 
associated with the error report lines that the error report was received on. It sets an internal flag 
and sets the Invalid bit in both of the error reports on the BERL and LERL lines that it is in the 
process of retransmitting (one Invalid bit for each error reporting line). 

2. If the Invalid bit of the incoming error report is set, it knows that error reporting error did not 
originate in one of this BXU's confinement areas. The BXU sets the Invalid bit in both of the 
error reports on the BERL 1 -BERL and LERL 1 -LERL lines that it is in the process of 
retransmitting (one Invalid bit for each line), but does not set an internal flag. 

In either case, the BXU participates in phase two of the error reporting if at least one report is error- 
free. An Error-Reporting-Error is marked as a permanent error, if it occurs within the permanent 
error window and in the same confinement area as the last Error-Reporting-Error. BXU's that detect 
error reporting errors do not participate in retry, but instead transmit the "Error Reporting Error" error 



12-15 



iny 



ERROR REPORTING 



report at the end of the transient waiting period. 

Fault-Tolerant Logic 

In addition to its error reporting network diagnostic capabilities, the BXU provides diagnostic 
support for the fault-tolerant logic. The following sections describe: 

the accessibility of the fault-tolerant logic 

the parity logic test 

FRC logic test 

bus time-out logic test 

the error reporting logic tests 

Control and Accessibility 

The control of which BXU in a logical module responds to fault tolerance diagnostic testing is 
provided by the Access bits in the FRC and QMR registers (see Appendix A for the description of 
the FRC register and the QMR register). Software sets the Access bits to point to the selected BXU. 

The accessibility to the fault-tolerant logic of the BXU is through the Test-Detection register and 
Access mode of logical IAC type 0010 B (Register Request Using a Logical Address). By writing a 
Testing enable bit to the Test-Detection register with a Logical IAC in the Access mode, only the BXU 
that the FRC and QMR registers access bits are pointing to is selected. This selection is important 
for parity testing. For instance, when parity testing is performed, the parity information must be 
corrupted for only one BXU in order to detect if the circuitry is properly functioning. By enabling 
the Access bit with this type of IAC, the software can specify any master or checker in the primary 
or shadow module to perform the test. In this case, the other BXUs in the primary and the shadow 
modules perform dummy operations to maintain lockstep operation. 

The Access bit can only be set if the IAC request is a Write Request to an AP-Bus register in another 
module. If an 80960MC processor sends an IAC to a BXU in its own module, the Access bit is 
ignored. Also, the Access bit will only control which BXU replies on the bus if both the toggle bits 
are off in both the FRC and QMR registers. 

Parity Logic 

The parity checking circuitry for each parity pin can be individually or jointly exercised by using the 
Parity-Test field of the Test-Detection register. The Parity-Test bits selectively force a parity error 
by inverting the CHK pin(s) sampled from the bus. This action will create a parity error, which the 
parity comparing logic should detect. The BXU reports an error if the Parity -Test bit is set and a parity 
error is detected. Parity testing can be performed on-line, but a real parity error (i.e., non-forced) 
during the test will go undetected. The success of the test is noted by checking the Error-Log register 
after the test. 

The Testing-Enable bit in the Test-Detection register must be set before executing parity testing. 



12-16 



iny 



ERROR REPORTING 



Before setting the Testing-Enable bit, there should be a single bit with a value of one in the COM 
register. This is a condition for any test that uses the Test-Detection register. 

FRC Logic 

The FRC circuits are self checking. To initiate FRC testing, software must perform the following 
actions: 

1 . Write a data word that contains a single bit with a value of one to the COM register. 

2. Set the Testing-Enable bit in the Test-Detection register. 

Once testing is started, the FRC circuits do not require any special test sequences to check their 
operation. The testing proceeds only as long as the Testing-Enable bit is set. 

Bus Time-Out Logic 

The bus time-out timer can be checked by performing an AP-Bus read operation to a memory location 
that has no BXU assig ned to it. A read request is used so that the 80960MC processor receives a direct 
indication (BAD AC asserted) that the time-out worked correctly. 

Error Reporting Logic 

The Test-Report command forces the BXU to test the error reporting network of the AP-Bus by 
generating an error report (see Appendix A for the description of the Test-Report command). The 
type of error report is specified by the bit pattern in the Test-Type register (see Appendix A for the 
description of the Test-Type register). Multiple errors may be loaded into the bit pattern to test the 
priority circuits in the fault-handling logic. After issuing the command, the software reads the Error- 
Log register to check that the error was properly handled. 

The Testing-Enable bit in the Test-Detection register must be set before executing the Test-Report 
command. Before setting the Testing-Enable bit, there should be a single bit with a value of one in 
the COM register. 

Handling Errors in the Fault-Tolerant Logic 

Errors may occur in the fault- tolerant logic. By latent fault testing, the BXU can detect these errors, 
if any, and respond to them appropriately. The possible errors that may occur in the fault-tolerant 
logic are listed below. 

Detection Mechanisms. If the FRC detection mechanism that is internal to the BXU fails while 
asserted, then the failure is handled correctly as an error. If this mechanism fails while not asserted 
(the internal drivers are open), then the failure is undetected. Latent fault testing can be performed 
by software to check for this failure. 



12-17 



WeT 



ERROR REPORTING 



Sensing and Encoding the Error Type. If this internal logic of the BXU fails to sense the error 
type properly or report the wrong error type while asserted, then the failure is handled correctly as 
an error. If the logic fails while not asserted (the internal drivers are open), then the failure is 
undetected. Latent fault testing is provided by using the Test-Report command and Test-Type 
register. 

Module-Error-ID and Bus-Error-ID Registers. A software-loaded parity bit detects any odd 
number of bit errors in the identification information sent on the AP-bus (see mechanism 1, which 
is described later in this section). Errors not detected by the parity bit could cause system failure. 
Latent testing can be performed using the Test-Report command and Test-Type register. 

Start Bit. If the Start bit fails while asserted, the failure is handled correctly (see mechanism 1 
below). If the Start bit fails while not asserted (the internal driver is open), then the failure is 
undetected. Latent fault testing can be performed by using the Test-Report command and Test-Type 
register. 

Error Report Generation. Bits that remain asserted are detected by using parity and signal 
duplication. The recovery from this situation uses mechanism 1 listed below. An open line causes 
an Error-Reporting-Error error message. 

Error Report Propagation. The error report flows through two totally in depend ent paths i n the 
BXU. Propagation errors are handled in the same manner as the errors on the BERL and LERL lines 
with internal fully redundant paths. 

Recovery Actions. Errors in recovery are caught by FRC disagreement between the master and 
checker. Latent testing is possible by forcing recovery actions to take place. 

Two special mechanisms are provided to detect and recover from some of the error cases described 
above. 



Mechanism 1. This mecha nism sp lits t he BER L lines from the master a nd chec ker to prevent a failed 
BXU f rom co rrupting both BERLj and BERL . The master only drives BERL and the checker only 
drives BERL 2 if any of the following conditions exist: 



• Neither BERL 1 nor BERL have a good report at the end of phase one or phase two. 

• The valid bit is not set in the Error-Log register at the beginning of retry 

• The valid bit is not set in the Err or -Log register when software writes to it (testing purposes only) 

This action allows th e other BXUs to issue an Error-Reporting-Error and prevents the faulty BXU 
from corrupting both BERL lines. This mechanism can be tested by clearing the Validhit and sending 
a test report. If the mechanism wor ks, then an err or -reporting- err or should be recorded (since the 
message would only go out on one BERL line instead of both of them). 

Mechanism 2. This mechanism prevents the system from suspending activity. An Unsafe- 
Confinement-Area type error report is issued, if the following two conditions exist: 
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• The current error report has an Err or -Rep or ting-Err or type associated with it. 

• The previous report contains an Error-Reporting-Error type. 

This mechanism avoids infinite loops alternating between a corrupt error reporting and the reporting 
of the corruption. 

SUMMARY 

Error reporting is the link between detection and recovery. The 80960 system uses a well defined 
protocol to report errors. This protocol defi nes the timi ng and error propagation methodology. The 
dual serial error reporting networks use the BERL and LERL lines to ensure that the error report is 
broadcast to every BXU in the system. The protocol takes into consideration the situation of multiple 
and simultaneous error reports. The error report format consists of a 50-bit message that incorporates 
redundancy and parity to ensure a correct report. 
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CHAPTER 13 
RECOVERY 

Previous chapters explained the 80960 architecture's mechanisms for detecting and isolating failures 
to a confinement area, and reporting those failures throughout the system. This chapter describes the 
architecture's hardware recovery mechanisms that are totally transparent to software. For example, 
when transient errors occur, the requests are automatically retried; when permanent errors occur, the 
hardware automatically removes the faulty module or bus, and reconfigures the system with 
redundant resources. 

The recovery algorithm is executed in parallel by all BXUs in the system. No global agent is 
responsible for correct recovery actions. Each BXU performs its recovery sequence independently 
from all other BXUs in the system. This distributed approach to error reporting ensures that no single 
component failure, as would be the case with a global agent, can bring the total system down. 

Figure 13-1 provides an overview of the error recovery procedure. Error detection and reporting, 
which were discussed in previous chapters, are included in the diagram for completeness. The 
recovery process encompasses the five major areas required for hardware fault recovery listed below: 

Types of Errors 
Unsafe Error Decision 
Retry Sequence 
Permanent Error Decision 
Resource Reconfiguration 

TYPES OF ERRORS 

The details of how these recovery actions operate are presented in separate sections. Before 
examining the recovery mechanisms, however, it is important to clearly define transient errors and 
permanent errors. Transient errors are often generated by electromagnetic noise bursts caused by 
some nearby device, such as a large motor starting or a relay de-energizing. Errors of this type require 
recovery (correction), but do not require a system hardware reconfiguration. However, permanent 
errors are indicative of a failed module or AP-Bus, and will require the hardware to isolate the 
defective confinement area and shift its activities to a back-up resource. 

Transient Errors 

An error is classified as a transient error, if both of the following conditions exist: 

The error type is not an Unsafe-Confinement-Area, Primary-Catastrophe, or Shadow-Ca- 
tastrophe 

• The error type or the confinement area in the current error report is different from the error type 
and the confinement area contained in the Error-Record register. (In other words, this is not the 
same as the previous error.) 
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Figure 13-1 : Recovery Procedure 
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Permanent Errors 

An error is classified as a permanent error, if either of the following conditions exist: 

The error type is an Unsafe-Confinement-Area, Primary-Catastrophe, or Shadow-Ca- 
tastrophe 

• The error type and the confinement area in the current error report are the same as the error type 
and the confinement area contained in the Error-Record register. 

When a permanent error is reported, the BXUs take special actions to recover from the failure. In 
all cases, the BXUs set the Permanent-Error bit in the Error-Log register. Additional recovery 
actions are taken by the BXU if the error occurred in its confinement area, or its backup/spouse' s area. 

UNSAFE ERROR DECISION 

The recovery sequence is the same for most error types except for the following three: Unsafe- 
Confinement-Area, Primary-Catastrophe, and Shadow-Catastrophe. These types of errors 
are handled immediately as permanent errors to prevent recursive error reporting cycles. For all other 
types of errors, retry is the first step in the recovery sequence. 

The BXU uses the Error-Log and Error-Record registers to make the unsafe error decision. The 
Error-Log register contains the current error report and the Error-record register contains the 
previous error report. If the error type is considered unsafe, either because it was one of the three 
immediately handled types described above or the two error registers match, then the BXU will 
directly enter the reconfiguration sequence. In all other cases, the BXU will begin the retry sequence. 

RETRY SEQUENCE 

The first step toward recovery is to retry the outstanding AP-Bus accesses. This section describes 
the retry sequence and explains some considerations as a result of the retry sequence. 

Retry Operation 

When an 80960MC processor makes a request, all information associated with the request (address, 
control, data) is automatically stored in the retry buffers of the BXUs. This information is held until 
the access is finished. All accesses that were outstanding at the time of an error will be retried. Storing 
the access request in retry buffers adds no delay to the access latency of an 80960MC processor 
request. 

If a retry sequence is necessary, the BXU must arbitrate again for access to the AP-bus. The 
outstanding requests of the BXU are retried in the same order as they were issued on the L-bus. This 
is true even if bus switch occurs because partner BXUs keep track of the state of requests being 
handled by their partners (see the "Bus Switching" section for details on bus switching). However, 
because the BXUs arbitrate for the AP-bus again, these requests may not maintain the same order on 
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the AP-bus with respect to requests issued by other modules. While the BXU retries it requests, it 
does not operate on new requests (an address is accepted, but the data is not processed). 

The retry buffers can be tested for correct operation by issuing a Test-Report command to any BXU 
in the system (see Appendix A for the description of the Test-Report command). If any buffer has 
failed, it will be detected as an FRC error in the module. 

Special Considerations for the Retry Function 

When the BXU retries requests, special considerations are made for three cases: completion of 
inbound operations, multi-word outbound read requests, and cache operations after a retry sequence. 

Completion of Inbound Operations 

Operations that are buffered by the BXU in the incoming direction are completed, regardless of 
retries on the AP-bus, if all data was received correctly into the buffer. These buffered operations 
include the following: 

• RMW-Read requests 

• Memory write operations if the BXU is acting as a memory controller 

In other words, once the data for these buffered operations is correctly received from the AP-bus (i.e., 
without errors being reported on the AP-bus BERL^BERLq lines), and the reply cycle is completed, 
the BXU finishes the operation on the L-bus. This process occurs no matter what happens on the AP- 
bus. 

For example, the BXU in Memory mode ensures that all write requests are atomic (either the 
transactions are completed or they are not performed) by fully buffering the write request before 
initiating any action on the L-bus. Once the L-bus operation is started, the BXU ensures that it is able 
to write the correct values into the memory in this module. 

Multi-Word Outbound Read Requests 

Multi-word outbound read requests may return inconsistent data if these requests are accessing 
locations that are being shared without the protection of hardware or software locks. 

The following example shows how a read request can return inconsistent data. 

Assume that the 80960MC processor performs a four- word read operation. The BXU accepts 
the read request and issues it on the AP-bus. The L-bus is held, by inserting wait states, until 
the data returns. 

• The data starts to return from the AP-bus and the BXU passes it onto the L-bus to maintain 
performance. The 80960MC processor accepts two words of the four- word request. 
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• An AP-bus error forces a retry and the read request is reissued by the BXU. 

• This time, however, the read request loses arbitration and a write request from another module 
writes to the same location that is accessed by the read request. 

• The retried read request now accesses updated locations and the last two words of the request 
are transferred to the 80960MC processor on the L-bus. However, the first two words are 
inconsistent with the last two words of the read data. 

In summary, single-word read operations are always indivisible with respect to write operations. 
However, multi-word read requests are not indivisible unless they are cached. 

Cache Considerations with Retry 

If an error is reported during a cache fill request, the cache write operation always is restarted from 
the beginning of the request after the retry sequence. Thus, if an error cancels an AP-bus reply after 
two of the four words were written into the cache, the two words will be rewritten when the request 
is retried. Starting the cache write operation from the beginning ensures that cache fill requests occur 
from a single AP-bus transaction. Consequently, the cache always holds consistent data. 

PERMANENT ERROR DECISION 

Permanent errors are defined in one of two ways: either the error type identifies the fault as a 
permanent error, or a transient error occurs twice in a row. Since the definition of a permanent error 
is receiving the same error report twice in a row, there needs to be a way to prevent the hardware from 
from labeling two errors that were hours apart as a permanent error. This is done with the Terminate- 
Permanate-Err or -Window Command. The permanent error window begins coincidently with the 
retry sequence. The period ends when software issues a Terminate-Permanent-Error-Window 
command (see Appendix A for the description of the Terminate-Perrnanent-Err or -Window com- 
mand). The BXU responds to this command by sending an error report using the Terminate- 
Permanent'Error-Window error type. Thus, the Terminate-Permanent-Error-Window error 
type provides the capability to set the time period in which the error can occur again. 

The Terminate-Permanent-Error-Window error type clears the Error-Count field in the Error- 
Log register in all BXUs. If Terminate-Permanent-Error-Window error report is received twice 
in a row, it is labeled a permanent error, and the normal permanent error operations occur (i.e., the 
module is removed from the system since the Terminate-Permanent-Error- Window error report 
uses the Module-Err or -ID register as its location identification). Because the Terminate- 
Permanent-Error-Window error report results from a software command, it can be used as a 
delimiter between error reports resulting from hardware error detection mechanisms. 

RESOURCE RECONFIGURATION 

Resource reconfiguration provides a way to recover from permanent errors. Once a faulty confine- 
ment area is known, the hardware system can automatically replace this area with a properly 
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functioning one, assuming that there are redundant resources available. To implement this, the 80960 
hardware system uses the following recovery mechanisms. 

1. Module shadowing 

2. Bus switching 

3. FRC splitting 

This section examines the details of each of these recovery mechanisms. Before explaining these 
items, however, the redundant resources are described to provide background information. 

Redundancy 

Recovery from permanent hardware failures can take place with redundant resources without any 
impact on logical software operation. With redundant resources, each of the self-checking modules 
and buses can be paired with an identical back-up module or bus. This feature, module shadowing, 
allows continued operation in the presence of any single failure. Modules can be paired with any 
other identical module. No predetermined electrical connections are required; shadowing is done by 
means of logical configuration information. The two modules in the pair then operate in lock step, 
providing a complete and current backup of all state information in the case of a failure. 

Buses can also be paired together to provide redundant bus channels. Since buses do not hold state 
information, both buses in a pair may be used to carry traffic during normal operation. Thus, the back- 
up bus capability can be used to increase total bus bandwidth. The mechanisms for establishing and 
maintaining redundant operation reside totally in the BXU. The 80960MC processor is unaware that 
it is operating in a redundant configuration. 

There is an important distinction between redundant resources and alternate resources. Redundant 
resources provide a complete and current backup for the resources in the system. When a failure 
occurs, the backup can mask the fault from the rest of the system. This recovery occurs automatically 
under the control of the hardware. Alternate resources provide multiple modules capable of 
performing the same tasks, but not a complete and current backup for the resources in the system. 
Alternate resources allow reconfiguration around failures, but do not mask the failure from the rest 
of the system. Alternate resources can be used to achieve deferred maintenance capabilities. An 
example of this type of configuration was given in Figure 10-7. 

The mechanisms and resources used for redundancy are orthogonal to the mechanisms and resources 
used for detection. Moreover, adding redundancy for recovery does not degrade the detection 
capabilities or performance characteristics of the system. 

Processor Module Shadowing 

The concept of module shadowing requires redundant resources in the form of two identical modules. 
The two modules must be operated in lock-step. One module is the active unit, or primary unit, and 
the other is the passive unit, or shadow unit. The module shadowing configuration is illustrated in 
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Figure 13-2. The BXUs in the primary and shadow units are paired together, or are considered 
"married". Each is the "spouse" of the other. 
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Figure 13-2: Primary/Shadow Modules with One System Bus 

Primary and shadow units can be married during initialization or during system operation using 
another module or special agent. Chapter 14 describes how to marry these pairs during initialization 
using a special agent. The following paragraphs show how to marry a primary and shadow units 
during system operation using another module. Because the marriage sequence is slightly different 
for a processor module and a memory module, each module is considered separately. 

Marrying Processor Modules Using Another Module 

Any two identical processor FRC modules may be married to form a shadowed module. The 
following steps need to be taken to marry processor modules: 

• Place the module (components and logic on the L-bus) in the same state. 

• Place the BXUs in the same state. 

• Make the primary and shadow units aware of each other by setting their Spouse-ID register equal 
to their respective Module-Err or -ID register (see Appendix A for the description of the Spouse- 
ID register). 

• Marry the primary and shadow units by setting the Married Bit in the QMR register. 

• Cause an event that synchronizes the primary and shadow units, such as sending a "Restart- 
Processor" IAC message. 
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Figure 13-3 shows an example of a married processor module configuration. In this example, all the 
components attached to the L-bus are placed in the same state (arbitrarily called state "ZZZ"). To 
ensure that the BXUs are in the same state, the registers of the QMR module are set to same value 
except those that are shown (the contents of each register are represented by the three capital letters). 
The Bus-Error-ID, Module-Error-ID, and Physical-ID registers of each FRC pair have different 
values to uniquely identify the FRC pair. The contents of the Spouse -ID register contain the contents 
of the respective Module-Err or -ID register of the FRC pair. These settings of the Spouse-ID register 
make each FRC pair aware of the other pair. The processor modules are married by setting Married 
bit of the QMR register in both FRC pairs and the Shadow bit of the QMR register only one FRC pair. 
The two married FRC modules form a single QMR logical module. 
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Figure 13-3: Example of Married Processor Modules 
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The following enumerated list is an example of one possible way (out of several) to marry the primary 
and shadow units. The module that becomes the primary unit is labeled the Primary- Elect unit, and 
the shadow unit, Shadow-Electunit. If there are multiple buses in the system, any action performed 
on one BXU in a module is performed on all BXUs in that module. 

1. Remove both processors from the pool of active processors available for work. 

2. Stop references to/from the components attached to the L-bus of the BXU in the Shadow-Elect 
unit by performing the following actions: 

a. Send the 80960MC processor(s) a "Stop-Processor" IAC message to stop the 80960MC 
processor(s) in the Shadow-Elect unit. 

b. Set the Shadow bit in the QMR register of the BXU, effectively making this the Shadow- 
Elect unit. 

c. Disable the AP-bus address recognizer of the BXU by clearing the AP-Match register. 

3. Ensure that the components attached to L-bus of the BXU in the Primary-Elect and Shadow- 
Elect units are in the same state by performing the following actions: 

a. Stop the 80960MC processor(s) in the Primary-Elect unit, effectively synchronizing the 
FRC pair. 

b. Disable the AP-bus address recognizer of the BXU's by clearing the AP-Match register. 

c. If there is local memory on the BXU's L-buses, it must contain the same data in the Primary- 
Elect md Shadow-Electmodules. This is done by copying the contents of one unit to the 
other. The Primary- Elect and Shadow-Electunits can ensure that memory has the same 
data by setting the AP-bus address recognizers to different values (both unused by the 
system currently in operation), then copying the contents of one memory into the other 
memory. At the end of this process, the address recognizers in both units are disabled by 
clearing the AP-Match registers. 

d. Clear the Toggle-Master/Checker bit in the FRC register in both units. This action allows 
the Toggle -Primary/ Shadow bit in the QMR register to be correctly set later in the marriage 
sequence. 

4. Synchronize the caches by sending an Invalidate -Cache command (see Appendix A for the 
description of the Invalidate -Cache command). 

5 . Send an error report of any type from any BXU. This operation synchronizes the deadlock timers 
of the BXU's in the Primary-Elect and Shadow-Elect units. 

6. Prepare the Shadow-Elect unit for marriage by using Physical Address IAC's to perform the 
following actions: 

a. Copy the contents of the registers in the Primary-Electpair to the registers of the Shadow- 
Elect pair except for the QMR, Spouse-ID, Module-Error-ID, Bus-Error-ID, Logical-ID, 
and Physical-ID registers. Because the Arbitration-ID register is a write-only register, 
software must write to both registers with the same value. 

b. Set the Module-Error-ID register in the Shadow-Elect unit to a unique ID. 
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c. Set the Bus-Error-ID register in the Shadow-Elect unit by setting the Source-ID field to 
a unique ID. 

7. Make units aware of each other by using Physical Address IAC's to perform the following 
actions: 

a. Copy the Confinement-ID field in the Module-Err or -ID register of the Primary- Elect unit 
to the Spouse-ID register of the Shadow-Elect unit. 

b. Copy the Confinement-ID field in the Module-Error-ID register of the Shadow-Elect unit 
to the Spouse-ID register of the Primary- Elect unit. 

8. Using IAC type 0100 B (Register Request Using a Physical IAC), copy the Logical-ID number 
of the Primary-Elect unit to the Shadow-Elect unit to synchronize the Primary-Elect/ 
Shadow-Elect pair. 

9. Marry the Primary-Elect/Shadow-Electunits, using IAC type 0010 B (Register Request Using 
a Logical IAC). By using this IAC type to access the QMR register, the same bits are 
simultaneously set in all BXUs. Marriage is accomplished by performing the following actions: 

a. If toggling between Primary- Elect and the Shadow-Electunits is desired, set the Toggle- 
Primary/Shadow bit in the QMR register in both modules. Otherwise, this bit is cleared in 
the modules. 

b. Set the Married bit in the QMR register of the Shadow-Electunit and Primary- Elect unit. 
This marks the modules as married. 

1 0. For an active/passive module, set up the married modules to recognize the same memory ranges 
by using Logical Address IAC's. The Logical Address IAC is used to simultaneously set up the 
Primary-Elect and the Shadow-Elect units. 

Address recognition is accomplished by performing the following functions: 

a. Set the AP -Match registers to the desired value. 

b. Set the AP-Mask registers to the desired value. 

c. Set the Enable bit in the AP -Match registers. 

1 1 . Activate the 80960MC processor(s) in the modules by sending a Restart-Processor IAC. 

If either module in the FRC pair fails, all the BXU's in the failed FRC module deactivate themselves, 
and their spouses take over operation. This allows recovery to be transparent to the rest of the system. 
The recovery procedpres are activated based on the error type and location information which was 
received by the BXU on the error report lines. 

The following steps show how to divorce the primary/shadow unit in a married processor module: 

1 . Stop the processors in the married pair by sending a "Stop-Processor" IAC message. 

2. Reset the Married bit in the QMR register. 

3. Change the ID's in the Arbitration-ID register. 
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4. Change the ID's in the Unit-ID field of the Logical-ID register. 

5. Start the processors of the resulting modules by sending a "Start-Processor" IAC message. 

Processor Module Recovery 

A primary/shadow pair automatically recovers from any single failure in the module confinement 
area. The recovery process for a permanent error in a module assumes that a primary and a shadow 
unit are available in the system. The shadow unit takes over if there is a permanent error in the primary 
unit, or the primary unit takes over if there is a permanent error in the shadow unit. An interrupt 
message can be sent to notify the operating system of the faulty module. 

An error is considered to occur in this module if any of the following actions occur: 

• The Confinement-ID field in the error report matches the Confinement-ID field of the Module- 
Error-ID register. 

A Primary-Catastrophe error report is received and the Shadow bit in the QMR register has 
a value of zero. 

• A Shadow-Catastrophe error report is received and the Shadow bit in the QMR register has 
a value of one. 

An error is considered to occur in this-module's partner if any of the following actions takes place: 

• The Confinement-ID field in the error report matches the Confinement-ID field of the Spouse- 
ID register. 

• A Primary-Catastrophe error report is received and the Shadow bit in the QMR register has 
a value of one. 

• A Shadow-Catastrophe error report is received and the Shadow bit in the QMR register has 
a value of zero. 

Actions by the Failed Module 

The failed module (FRC Pair) isolates itself from the AP-bus and ignores all requests from the L-bus. 
Requests from the AP-bus are ignored, except for IAC access type 0100 B (Register Request using 
a Physical Address) and IAC access type 01 1 1 B (Identify Device IAC). The IAC requests using a 
physical address can be used by diagnostic software to probe into the failed module. If another 
permanent error is reported in this module, however, then the IAC access type 0100 B is also disabled. 

On receiving an error report indicating a permanent error in this module, all BXUs in the module take 
the following actions: 

• The BXUs set the Faulty bit in the FT2 register (see Appendix A for the description of the FT2 
register). This disables the AP-bus recognizers for memory addresses, IAC messages (IAC 
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access type 001 1 B ), and IAC requests using a logical address (IAC access type 0010 B ). The AP- 
bus recognizers for IAC requests using a physical address (access type 0100 B ) and Identify 
Device IAC (access type 01 1 1 B ) remain enabled. The L-bus recognizers for all IAC and memory 
addresses are disabled, however. When the Faulty bit is set, the BXU also drives the COM pin 
low. Under this condition, the BXU can still generate error reports and will accept error reports 
from the BERI^-B ERI^ l ines, but it will not propagate error reports from the LERLj-LERLq 
lines to the BERL r BERL lines. 

• The BXUs reset the Married bit in the QMR register. 

If the Faulty bit was set before this error report was received, then the BXUs set the Inactive bit 
in the FT1 register. This disables the AP-bus recognizers for the memory addresses, the IAC 
messages, and the IAC requests using physical or logical addresses (the Identify Device IAC 
requests are accepted). These actions totally isolate the module from the AP-bus. The BXU does 
not generate error reports, nor does it propagate error reports from the BERLj-BERL lines to 
the LERL 1 -LERL lines. The BXU does not accept error reports from BERLj-BERL lines 
unless the BXU is in Memory mode (the BXU -Mode bit in the LBI-Control register is zero). The 
Inactive bit in the FTI register is set conditionally to allow diagnostic software to have access 
to the BXU, if at all possible. 

Actons by Partner Module 

If a module's partner fails, the surviving module takes over operation. On receiving an error report 
indicating a permanent module error in the partner module, all BXUs in this module reset the Married 
bit in the QMR register. This causes this module to respond to the AP-bus requests. 

Memory Module 

A memory module may consist of a BXU (in Memory mode), a memory controller, and a RAM array, 
as shown in Figure 13-4. This section describes the response to an error report, the ECC interface, 
memory module shadowing, and memory module marriages. Except for correctable and uncor- 
rectable array errors, memory module recovery is identical to processor module recovery, which was 
covered in the previous section. 

Response to Error Reports 

When an error report occurs, the data stops flowing through the BXU. If the BXU was in the process 
of driving a write request on the L-bus, then the remaining data cycles are invalid and the BE 3 -BE 
lines are not asserted. During the transient waiting period, the entire write request is reissued on the 
L-bus. This ensures that the memory array holds valid data before retry begins. If the BXU was 
performing a read request at the time of the error report, the L-bus cycle is completed, but the data 
is not used. 

A BXU in Memory mode sets all of its RMW-Lock bits in the Lock register on every error report. The 
lock timer begins at the end of the transient waiting period. 
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Figure 13-4: Memory Modules 

Memory Controller Interface 

The BXU supports discrete memory controllers that provide single error correcting and double error 
detecting ECC protection for the memory ar ray. An ECC error signal from an external controller 
causes the BXU to assert the BERLj-BERL lines with the same timing as the AP-bus parity bits. 
Thu s, ECC errors a re handled correctly even if the BERL-Wait bit in the FT I register is disabled (see 
the "BERLj-BERLq Timing" section in Chapter 12. T he interface between the BXU and the external 
ECC l ogic consists of three signals: Correct (COR), ECC Error (ECC), and Uncorrectable ECC 
(UNC). 



COR 



The Correct signal is used by the BXU to inform the external memory 
controller to correct the memory data as it flows onto the L-bus. If this signal 
is not asserted, then the memory data ma y flow directly onto the L-bus with 
only error checking, but no correction. COR is asserted at lea st one cycle 
before the next memory address is sent on the L-bus. If COR is asserted, 
then the Fast-Reply bit in the Lock register is over-ridden, and the data is 
buffered before going on the AP-bus. 



ECC 



The ECC Error input signal indicates that an error was detected from the 
external ECC logic. ECC is asserted if the ECC logic detected an error in 
the memory data. This signal may be asserted even though the external logic 
may be correcting t he erro r and providing correct data on the L-bus. If the 
BXU is asserting its COR signal, the ECC signal is ignored. Only the UNC 
pin is checked for an error indication under these conditions. The ECC 
signal must be valid one cycle after the data word is transmitted. 
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UNC The Uncorrectable ECC input signal is used by the external ECC logic to 

indica te to the BXU that it detected an uncorrectable memory error. The 
UNC signal must be valid one cycle after the data word is transmitted when 
the ECC Controller is in the correct mode . 

If a permanent uncorrectable ECC error is detected, then the BXUs shut down the entire memory 
module. 

Often the most desirable design of a memory module requires access to information located in the 
memory controller. For example, the external memory controller can provide registers that record 
ECC errors. The BXU provides an access to these registers by mapping 16 IAC register locations 
to the memory controller. 

When the BXU receives an IAC request to these locations, i t map s the request onto the L-bus. When 
the request is issued on the L-bus, the BXU holds the MEM/REG line low during the Ta cycle on the 
L-bus. This conveys to the memory contr oller th at the request is for its control registers rather than 
the memory location. While the MEM/REG line is low, the BXUs also disable their L-bus 
recognizers to ensure that the I AC on the L-bus is not accepted by another BXU. Thus, this register 
access facility, along with the COM pin, and the COM register of the BXU can assist software to 
initialize the discrete memory controller. 

Restoring Failed Memory Locations 

The external memory controller must provide a mechanism for restoring correct values to memory 
locations that have single bit errors. This could be a scrubbing mechanism during the refresh cycles 
or it could be a special diagnostic memory access. The only constraint is that it must be done 
atomically (read, correct, and restore without interruption) and in lockstep even though only one of 
the modules may have an error. 

Failures in Partial Write Operations 

If a partial write access encounters an address location that has an uncorrectable error, it must leave 
the location in a state that causes subsequent read accesses to generate an uncorrectable error. Any 
other words that are written as part of the partial write operation must be written completely. At the 
completion of the partial write operation, all locations must have their new data values except the 
faulty location, which returns an uncorrectable error on any subsequent read operations to the 
location. 



Note that this means that the controller cannot assert either the UNC or the ECC signals. Asserting 
these signals generates an error report. Because the BXU issues the AP-bus reply before the write 
operation has completed, there is no guarantee that the request will be retried. Thus, the memory 
could remain in an inc orrect state. If the memory controller wants to abort this partial write operation, 
it must assert the ERR signal. This causes the BXU to generate an Unsafe-Confinement-Area error 
report that immediately shuts down the entire module. 
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Memory Module Shadowing 

Most of the facilities required to support memory module shadowing are the same as for processor 
module shadowing. The only additional requirement is that the refresh function of the two modules 
must operate in lockstep. 

To accomplish this, the BXU uses the Force Refresh (FRF) signal that is activated by the Sync- 
Refresh command (see Appendix A for the description of the Sync-Refresh command). The FRF 
signal is an output from the BXU that informs the external memory controller to immediately perform 
a refresh function. 

When the BXU receives a Sync-Refresh command, it generates an error report with error type Sync- 
Refresh. If the BXU receives a Sync-Refresh error report and if the Confinement-ID field matches 
either the Confinement-ID field in the Module-Error-ID register or the Spouse-ID register, then the 
BXU asserts the FRF pin for one cycle. In other words, receiving a Sync-Refresh error type with 
its own or its spouse's module identification causes the BXU to assert the FRF signal for one cycle 
at the beginning of the transient waiting period. 

The external memory controller uses the FRF signal to force one or more refresh cycles in such a way 
that the two modules have refresh cycles operating in lockstep. An error report is used because all 
traffic to the modules is stopped. Thus, the refresh commands have exactly the same timing in both 
modules. If this error report is received twice in a row, however, it would be handled as a normal 
permanent error (shutting down the module). 

Bringing up a memory module shadow when one of the modules is actively providing data to the 
system places restrictions on system operation. The only way to copy the data is to perform standard 
memory read and write operations. 

Memory Module Marriage 

The following steps, which are similar to those of the processor module, need to be taken to marry 
two memory modules: 

• Place the module (components and logic on the L-bus) in the same state. 

• Place the BXUs in the same state. 

• Make the primary and shadow units aware of each other by setting their Spouse-ID register equal 
to their respective Module-Error-ID register. 

• Marry the primary and shadow units by setting the QMR register. 

• Cause an event that synchronizes the primary and shadow units, such as sending aSync-Refresh 
command. 
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The memory refresh synchronization of the primary and shadow units is the basic difference between 
marrying memory modules and marrying processor modules. The Sync -Refresh command causes 
the external memory controller to be synchronized to the system clock. When using this command, 
the Toggle-Primary I Shadow bit in the QMR register must first be cleared. After using this command 
and resetting the Silent bit in the QMR register, the Toggle-Primary/Shadow bit may be set. 

Bus Switching 

To recover from permanent AP-bus errors, at least one additional AP-bus must be available. Figure 
13-5 illustrates a dual-bus Primary/Shadow pair. Note that each module is attached to two buses 
(i.e., it is dual ported). From the viewpoint of a given BXU, the system bus that it is attached io^is 
its Primary bus, and the other bus is its backup. 
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Figure 13-5: Primary/Shadow BXUs with Two AP Buses 

The BXUs in a system are capable of automatically switching AP-buses when a permanent error 
causes one AP-bus to stop functioning. When a permanent error is detected, all the BXUs on the 
faulty AP-bus isolate their modules from this bus. The BXU on the failed AP-bus ignores L-bus 
requests, while the BXU on the backup AP-bus picks up these requests. The data that normally would 
have flowed on the failed bus, now flows on the backup bus. If a BXU has a cache, the BXU 
invalidates its cache directory because the directory must be reorganized to match the new (and 
larger) address space, including a new interleaving factor. 

During normal operation, all AP-buses in the system may be used to transmit data because the AP- 
bus does not hold state information. AP-buses do not operate in lock-step (FRC), thus the data on 
each individual bus are unique packets. This multiplicity of AP-buses increases the system 
throughput without sacrificing back-up capability.. Normally the memory address range is 
interleaved between the two buses, but this is not required for bus switching. 
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While processor modules are connected to all active AP-buses, a memory module connects only to 
the AP-bus with the corresponding address range. It also connects to the backup bus, although this 
connection is normally inactive. Memory-mapped I/O modules, in general, connect to all buses in 
interleaved systems. Should the primary bus fail, the BXUs that connect the memory module to the 
failed bus isolate themselves from the failed bus, and the BXUs on the backup bus become active and 
begin receiving requests. Figure 13-6 shows an example of bus switching. 
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Figure 13-6: Permanent Bus Error Recovery 

The only bus pairings that are allowed are AP-bus and AP-buSj, and AP-bus 2 and AP-bus 3 . This 
restriction is required because recovery occurs by changing the interleaving factor. Because of this 
restriction, there is no need to make the BXUs aware of their backup bus (they always know the one 
and only bus that can backup their bus). Setting up a bus pair is done with the following steps. For 
each step, the BXUs on both buses must be setup. 

1 . The Function bits in the Match registers must be set to a compatible set of values (see Appen- 
dix A for more details on the Function bits). 

2. The Bus-Switching-Disable bit in the AP -Control register must be cleared. 



Clearing t he Bus-Switch-Disable bit in the AP -Control register allows a special use of RPYDEF. The 
RPYDEF signal is used only to disable the bus time-out. No reordering of the bus pipeline occurs. 



Bus switching provides recovery for memory access and IAC Messages. IAC register requests that 
are addressed to BXUs on the failed bus will return bad-access replies (because the request will have 
been routed out on the backup bus). The BXUs on the failed bus are still accessible to the processors 
on their L-bus by using IAC requests. 

The remaining parts of this section describe the details of the bus switching mechanism and present 
the system implications of using this mechanism. 
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Bus Recovery 

When an AP-bus fails, all traffic is routed to the backup bus for a system with paired AP-buses. This 
section describes the details of how this is done. From the viewpoint of a given BXU, the AP-bus that 
it is attached to is the Primary Bus, and the other bus is its backup. 

An error can occur either on the primary AP-bus or the backup AP-bus. A failure is considered to 
occur on the primary AP-bus if one of the following actions takes place: 

• The Unsafe-Confinement-Area error type occurs and the Confinement-ID field in the error 
report matches the Confinement-ID field of the Bus-Error-ID register. 

• The Error-Reporting-Error, Bus- Arbitration, Bus-Parity, Attach-Bus, or Detach-Bus 
error type occurs twice in a row in the same permanent error window and the Confinement-ID 
field in the error report matches the Confinement-ID field of the Bus-Error-ID register. 

• The Detach-Bus error type occurs and the Confinement-ID field in the error report matches all 
bits of the Confinement-ID field of the Bus-Error-ID register except bit 8 (this is the backup bus 
confinement identification). 

A failure is considered to occur in the backup bus if one of the following actions takes place: 

• The Unsafe-Confinement-Area error type occurs and the Confinement-ID field in the error 
report matches the Confinement-ID field of the Bus-Error-ID register except bit 8 (this is the 
backup bus confinement identification). 

• The Error-Reporting-Error, Bus-Arbitration, Bus-Parity, Attach-Bus, or Detach-Bus 
error type occurs twice in a row in the same permanent error window and the Confinement-ID 
field in the error report matches the Confinement-ID field of the Bus-Error-ID register except 
bit 8 (this is the backup bus confinement identification). 

• The Detach-Bus error type occurs and the Confinement-ID field in the error report matches all 
bits of the Confinement-ID field of the Bus-Error-ID register. 

Actions by the Failed Bus 

All BXUs on the failed AP-bus isolate their modules from the faulty bus. Consequently, requests 
from the faulty bus are ignored. L-bus requests for this AP-bus are ignored by the failed BXU, and 
are picked up by its partner. The failed BXU, however, accepts its L-bus IAC requests and responds 
to them. 

On receiving an error report indicating a permanent bus error on this AP-bus, all BXUs on this bus 
take the following actions: 

• The BXUs set the Inactive bit in the FT1 register. This disables the AP-bus recognizers for 
memory addresses, IAC message (IAC access type 001 1 B ), IAC requests using a logical address 
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(IAC access type 0010 B ), and IAC requests using a physical address (IAC access type 0100 B ). 
The BXUs still recognize the Identify Device IAC (access type 01 1 1 B ). 

• The BXU does not accept error reports from the BERLj-BERI^ lines unless the BXU is in 
Memory mode. The L-bus recognizers of the BXU are disabled for memory addresses. The 
BXU accepts only IAC requests addressed to it. No IAC requests are accepted that are for 
another BXU on the failed bus, however. The BXU stops tracking and buffering requests 
handled by its partner, and begins tracking and buffering requests from the L-bus that would 
normally have been routed through this BXU (but are now being handled by the backup BXU). 

• The BXUs invalidate their cache directory. 

• The BXUs clear the Enable, I/O * -Channel - Active, and L '0 ] -Channel } - Active bits in the Prefetch- 
Control register to disable the I/O prefetching function. 

Actions by the Backup Bus 

All BXUs on the backup bus pick up the traffic that normally would have flowed onto the failed bus. 

Upon receiving an error report that indicates a permanent error in the partner's bus, all BXUs on this 
bus take the following actions: 

• The BXUs set the Backup-Bus-Inactive bit in the FT2 register. This causes several actions to 
occur: 

— The BXU retries all requests that it buffered internally, not just its own requests. 

— Bus requests that were pending at the time of the error are sent on this AP-bus. 

— The L-bus memory address recognizers ignore LAD 4 if they were interleaved, or become 
active if they were non-interleaved and considered a backup (see Appendix A for the 
definition of the Function bits of the Match and Mask registers ). 

— The IAC address recognizers match on all requests that would have flowed onto the failed 
AP-bus. 

— The new interleaving information is sent to the cache logic to update its address mapping 
logic. 

• The BXUs invalidate the cache directory. Because this directory now handles a larger address 
space, the cache directory is reorganized to match the new address space (for instance a different 
interleaving factor). Before reorganizing the directory, the old one is invalidated. Because the 
cache uses a write-through update policy, this invalidation can be done simply in the BXU. 
There is no need for additional updates to memory. 

• The BXU clears the Enable, HO -Channel - Active, and II O -Channel { - Active bit in the Prefetch- 
Control register. This disables the I/O prefetch function. This action prevents errors due to the 
potential change in interleaving factor (the data returned from the prefetch buffers might be the 
incorrect data). All requests that normally would have been handled by the prefetch unit are 
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handled as a normal AP-bus memory request. 

Requests to a Failed Bus 

Requests to a failed AP-bus can result in one of several replies listed below: 

• If the request is to the BXU on the processor's L-bus, then the BXU processes the request and 
replies correctly. 

• If the request is to a BXU in another module, then the request (either memory or IAC) is sent 
to the backup AP-bus (assuming that a backup AP-bus exists). This action results in a normal 
reply for a memory request or a message IAC (access type 001 1 B ). For an IAC requests that uses 
a physical or logical address (access types 0100 B or 0010 B ), the bad-access reply is used because 
the IAC address specifies a BXU on a failed bus. 

• If the request is to a BXU in another module and there is no backup AP-bus, then the request 
suspends activity on the processor's L-bus. Because no BXU on the L-bus recognizes the 
request, no BXU asserts the READY signal. 

IAC Operations After a Bus Switch 

After a bus failure and bus switch, IAC register requests that normally would have used the failed 
bus, are routed to the backup bus. These requests return bad-access replies because no BXU on the 
backup bus matches the Bus-Destination field in the IAC address. The BXUs on the failed bus may 
still be accessed from their L-buses. IAC requests to the local BXUs are not mapped to the backup 
bus. 

If the failed bus was the message bus (AP-bus ), then the backup bus (AP-bus^ takes oyer as the 
message bus. All IAC messages are now handled over the new bus. This switch is transparent to 
the 80960MC processor. The backup bus (AP-bus^ can also become the message bus if a Detach- 
Bus command is issued for the message bus confinement area. If this command occurs, an Attack- 
Bus command can be issued for the original message bus confinement area making AP-bus the 
message bus again. 

The next few paragraphs show the details of how the message bus switching operates. BXUs that 
are not currently the message BXUs track the actions of the message BXUs in the following way. 

• Write operations to the Processor-Priority register are done in all BXUs on the L-bus using the 
"Register Request From the L-Bus" IAC (access type 0000). Consequently, all Processor- 
Priority registers always have the most recent priority information (see Appendix A for the 
description of the Processor-Priority register). 

• The Message-Buffer -Full bits in the Processor-Priority register track the status of the IAC lines. 
If IAC is asserted, then the appropriate Message-Buffer -Fullbit is set in all non-message BXUs. 
If IAC is not asserted, then the appropriate Message-Buffer-Full bit is cleared. Thus, the non- 
message BXUs always contain the most recent status of the message buffer registers. 
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• The Message-Data-Valid bit in the Processor-Priority register remains cleared in the non- 
message BXUs. 

As a result of a bus switch, the backup bus BXU becomes the message BXU. Because the backup 
BXU was tracking the message state information, it is prepared to immediately respond to IAC 
message requests from the AP-bus. If an IAC message was waiting for the 80960MC processor 
before the bus switch, then the Message-Data-Valid bit in the original message BXU is set, and the 
Message-Buffer-Full bit in all the BXUs are set. When a message-buffer read request is issued, the 
BXU with the associated Message-Data-Valid bit set replies with the data. Thus, if a bus switch 
occurs after an IAC message was received, but before the 80960MC processor read the message 
buffer, the IAC message is correctly passed on to the 80960MC processor. 

Attach-Bus Command 

The Attach-Bus and Detach-Bus commands can be used to switch the message bus or to perform 
diagnostics on the AP-buses. For example, if the 80960MC processor issues a Detach-Bus 
command, the BXU responds by issuing a Detach-Bus error report, which removes the bus from 
operation and causes the backup bus to handle all the requests. After tests verify correct operation 
of bus switching, the Attach-Bus command connects the AP-bus back to the system. The BXU 
responds by sending an Attach-Bus error report. 

The Attach-Bus error report establishes operation on a pair of buses again. The following sequence 
shows how to attach a bus: 

• Set the Sequence bit in the Match register of every BXU on the pair of functional buses. (The 
Sequence bit in the Match registers of the BXUs on the other bus pair of a four bus system do 
not need to be set.) The Sequence bit is set to ensure that there is no more than one access pending 
from the L-bus to this AP-bus pair. 

• Clear the Enable, I/O > -Channel - Active, and II O -Channel ^ Active bits in the Prefetch-Control 
register in every BXU on the pair of functional buses. This prevents the prefetch unit from 
generating additional accesses on the bus that might prevent the correct switching. 

• Wait approximately 100 cycles after the last Sequence bit is set. This action ensures that all 
modules are operating in sequential mode. 

• Send an Attach-Bus command to one BXU on the functional bus of the pair. This command 
causes an Attach-Bus error report to be generated with the Location-ID field equal to the Bus- 
Error-ID register of that BXU (the Confinement-ID field will match the Bus-Error-ID register 
of the currently operating bus in the pair). 

The response to the Attach-Bus command depends on whether the bus or its backup bus was 
removed. If the Confinement-ID field matches the Bus-Error-ID register, then clear the Backup- 
Bus-Inactive bit in the FT2 register. This action returns the L-bus address recognizers to their normal 
state. If the Confinement-ID field matches the £ws-£rr0r-/D register except for bit 8 (i.e., the backup 
bus), then clear the Inactive bit in the FT I register. This action returns the L-bus address recognizers 
to their normal state. 
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Both BXUs in the bus pair in each module re-evaluate the address of the one request that may be 
outstanding in the BXUs and decide whether they should retry the request. For an outstanding read 
operation, the BXU finishes the request during retry on the AP-bus that normally handled this 
operation when two AP-buses were available. For an outstanding write operation, the BXU finishes 
the request during retry on the AP-bus that handled the request when only one AP-bus was available 
(even if two AP-buses are available now). 

If an Attach-Bus error report is received twice in a row, it is labeled a permanent error, and the normal 
permanent error operations will occur (i.e., the bus is removed from service since this report uses the 
Bus-Error-ID as its location identification). 

Communication Between Buses 

When a BXU determines that its backup bus has failed, it retries any outstanding requests of the BXU 
attached to the failed bus, as well as its own requests. As with retry sequence without bus switching, 
the requests from one module are issued in the same order as they were received on the L-bus. The 
internal queuing mechanism in the BXU allows it to combine the requests from the two buses in the 
correct order. 



The BXUs within a module use two signals, Pop Queue (POPQUE) and Subsyste m Busy (SS BUSY), 
to coordin ate the activity between the AP-buses. Figure 13-7 shows how the POPQUE and the 
SSBUSY signals are connected. 



CPU 

"77 



CPU CPU 

7_l L_ 



P5?q"ue * 



-K BXU 
■V 



POPQUE 



BOUT 



♦"■modchk 1 -*- 



AP-BUS 



CPU 

7T 



I 



♦-■modchk 1 -*- 



E 3 



Figure 13-7: System Connection to POPQUE and SSBUSY 
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The pair of BXUs use the POPQUE signal to inform each other of the state of pending requests on 
their AP-bus. Since read requests and IAC requests hold up the L-bus until the data is returned, only 
write memory requests are monitored. Although only one BXU actually performs the write 
operation, the write requests are buffered in both BXUs associated with the pair ed AP-bus es. When 
the operation is completed on the AP-bus, the partner BXU is informed by the POPQUE pin. This 
signal is asserted when the write request that has been in the pipeline queue the longest time is 
completed. 



The internal queuing and POPQUE signaling protocol ensures that every request issued by the 
80960MC processor on the L-bus is serviced once and only once by the addressed component on the 
AP-bus. During the transient wait period a different protocol conveys the status of all write requests. 



The SSBUS Y signal connects all the BXUs in a module (that are in the same subsystem). When this 
signal is asserted (low), the BXUs accept the address of the request, but do not transfer data. This 
signal ensures that the B XUs handle RMW- Write requests, IAC requests, and retry operations 
correctly. The SSBUSY signal ensures that all the BXUs are not busy handling IAC requests or 
performing prefetch operations. 

Memory Range Recognition Considerations 

The BXU provides address recognizers that can be individually programmed for interleaved or non- 
interleaved accesses to the AP-bus. The simplified diagram of Figure 13-8 illustrates how the BXU 
effects recovery for three cases: range recognition only (no interleaving), bus interleaving only, and 
both range recognition and bus interleaving. 
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Figure 13-8: Bus Recovery Illustration 
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Case 1 : Range Recognition Only (No Interleaving) 

The address recognizers in BXUs A and B are set for no interleaving. To recover from a permanent 
bus error on AP-bus , the following must be true: for every Match register in BXU A that has its 
Function bits set to 10 B , the corresponding Match register (see Appendix A for a description of the 
MATCH register) in BXU B must cover the same range and have its Function bits set to 1 1 B (and 
vice versa). 

In normal operation requests only flow through BXU A. In case of a bus switch, the address 
recognizers of BXU B are activated and begin to pick up the traffic that was previously flowing 
through BXU A. 

Case 2: Bus Interleaving Only 

The address recognizers in BXU A and BXU B are set to interleave between AP-bus Q and AP-bus r 
A permanent bus error on AP-bus is recoverable under the following conditions: the Function bits 
in all Match registers are set to 01 B , and the two BXUs have complementary interleaving factors set 
up in the LBI-Control register (the Mask bits should be the same and the Match bits should be the 
inverse of each other in the low order bit). 

Recovery is accomplished by changing the interleaving factor. This change in interleaving affects 
the L-bus address recognizers and forces the BXU to recognize different address bits after recovery. 
Table 13-1 illustrates the address bits that the BXU interleaves before and after a bus error forces 
a bus switch. Note that the total memory interleaving (i.e., the combination of bus and memory 
controller interleaving) remains constant through the recovery sequence for any of the modes. 

Table 13-1: Bus Recovery Effects on Interleaving 



Interleaving 


Address Bits Used for Interleaving by the BXU 


Total 


Bus 


Before Bus Failure 


After Recovery 


1 


1 


— 


— . 


2 


1 


— 


— 


2 


2 


AD 4 


— 


4 


2 


AD 4 


— 


4 


4 


AD 5 and AD 4 


AD 5 



Case 3: Both Bus Interleaving and Range Recognition 

This configuration is allowed and is achieved by setting up the various Match registers in the manner 
described for the two earlier cases. 
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Memory Considerations 

The bus switch for a memory module is the same as for processor modules, plus one added operation. 
The BXU on the surviving bus of the bus pair sets all of its RMW-Lock bits in the Lock register. This 
action ensures that any RMW-Lock bit that was set before the bus failed is honored correctly. 
Although some RMW-Lock bits may be set unnecessarily, these extra bits are cleared by the lock 
time-out. 

Cache Considerations 

As a result of a bus switch (permanent bus error, Detach-Bus command, or Attach-Bus command), 
all BXUs on the paired AP-buses invalidate their entire cache directories and set the Line- 
Configuration bit in the Cache-Configuration register (forcing eight lines per block, which is the 
only non-interleaved configuration that is supported). This invalidation is necessary to prevent 
incorrect mapping of data in the cache. 

The remaining BXUs establish their directories again using the new configuration. Table 1 3-2 shows 
the change in the cache configuration as a result of bus switching actions (see the Cache- 
Configuration register described in Appendix A for definition of configuration numbers). The cache 
remains enabled, and the memory requests that follow the bus switch refill the cache. In a four AP- 
bus configuration, the BXUs attached to the other paired bus do not invalidate their cache 
configuration. 

Table 13-2: Cache State Change Caused by Bus Switch 



Configuration Before Failure or 
After Attach-Bus Command 



Configuration After Failure or 

After Detach-Bus Command or 

Before Attach-Bus Command 



Failure or Detach-Bus Command 

-< — 



Attach-Bus Command 



Four-way interleaved (LAD 4 and 
LAD 5 ), configuration #1 or #4. 



Two-way interleaved (LAD 5 ) 



Two-way interleaved (LAD 4 ), 
configuration #2 or #5. 



No interleaving, configuration #3 



No interleaving, configuration #3 



Off 



NOTES: 

1. The configuration numbers are from the description of the Cache-Configuration register. 

2. These changes are true only for the surviving bus in the bus pair. If there is another pair of buses 
(i.e., 4 buses total), the other BXUs do not have any configuration change. 

In four-way interleaved systems, the cache size is reduced to five-eighths of its original size (four- 
eighths available for the operational bus in the pair and one-eighth available for the surviving bus in 
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the failed pair). Although the size of the cache is reduced, all SRAM locations are used. This sizing 
occurs because of the double mapping of the address bits (some bits are used by the cache directory 
as tag bits and by the SRAM as address bits). 

In a two-way interleaved cache, the cache size is reduced to one-half of its original size. 

I/O Prefetch Considerations 

A bus switch affects BXUs that perform I/O prefetch functions. When a bus switch occurs, both 
prefetch channels in the two BXUs (on the failed and surviving AP-bus) are disabled by resetting the 
Enable bit in the Prefetch-Control register. This is done because the changes in interleaving factors 
may invalidate the data buffers of the prefetch unit. Data requests are still correctly serviced. 
However, the requests flow to AP-bus memory rather than being serviced from the prefetch buffer. 
Thus, the access time of the requests are slower as a result of the bus switch. 

Before the prefetch channels can be used again, software must set the Enable bit. If a Start command 
is issued while the prefetch Enable bit is zero, the command is treated as a no operation. 

After a bus switch, outstanding prefetch read requests are lost by the BXU that connects to a failed 
AP-bus. Consequently, the processor requesting the prefetched data experiences a longer latency for 
these read requests with address locations covered by the quiescent BXU. This condition will exist 
until the end of that transfer, but the re-initialization of either prefetch unit in the other BXU before 
the start of the next transfer will bring operation back to normal. 

FRC Splitting 

FRC splitting is a function provided by the BXU to allow systems to recover after a fault when module 
shadowing is not used. It allows recovery by splitting a master/checker pair so that one component 
continues to run while the other shuts down. FRC splitting can be used in applications that need 
enhanced reliability, but where the consequences of a fault are not severe. FRC splitting minimizes 
the number of resources required to achieve fault tolerance, but recovery is not transparent to 
software, proper identification of the faulty module is not ensured, and the system is not fault- tolerant 
after an FRC split. FRC splitting requires a single arbitration network shown in Figure 1 1-4. 

FRC splitting occurs only with a RESET sequence. This form of recovery is enabled in all BXUs 
after a cold RESET. It may be disabled by software by using the FRC -Splitting-Disable bit in the 
FRC-Splitting-Control register. Some methods to reset an 80960 system are listed below. 

• A software command can prompt external support logic to send a local RESET 

• A watchdog timer can start the RESET sequence when the system fails 

• A person can manually reset the system by pushing a button 

This form of recovery is not a result of the error report. FRC splitting occurs as part of a warm RESET 
sequence. Consequently, this recovery approach is not transparent to the user or the software system. 
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Because FRC splitting eliminates the FRC error checking in the module, diagnostic software plays 
a role in the trial and error sequence that determines which half of the module is operating correctly. 

FRC splitting occurs if the following conditions exist: 

• A warm RESET occurs 

FRC-Splitting-Disable in the FRC -Splitting-Control register is cleared 

• My -Permanent-Error bit in the FRC -Splitting-Control register is set 

The My -Permanent-Error bit in the FRC -Splitting-Control register is set if the Confinement-ID field 
in the error report matches the Confinement-ID field in the Module-Err or -ID register, and the 
Married bit in the QMR register has a value of zero. 

The BXUs in the FRC pair respond to the permanent error in the same manner as described in the 
"Processor Module Recovery" section. The faulty module is removed from the system, but the 
BXUs in the faulty module can execute FRC splitting on the next warm RESET, if that form of 
recovery is enabled. 

If FRC splitting occurs, the Separated-Master I Checker bit in the FRC -Splitting-Control register is 
set. The BXU selects either the master or checker as the active component. When FRC splitting first 
occurs, the master is the active component, on a subsequent warm RESET the components toggle 
the active state. For example, the checker will be active on the second, fourth, etc. assertion of RESET 
and the master is active on the third, fifth, etc. assertion of RESET. The state of the two BXUs is 
shown Table 13-3. 

Table 13-3: FRC Splitting Control Bits State 



Control Bit (Register) 


Original State 


State on First, 
Third, ...Reset 


State on Second, 
Fourth, ...Reset 


Master 


Checker 


Master 


Checker 


Master 


Checker 


Active-Component 


X 


X 


1 








1 


Master (FRC) 


1 





1 





1 





Toggle-M-C (FRC) 


X 


X 














Passive (FSC)* 











1 


1 





Separated-M-C (FSC)* 








1 


1 


1 


1 



NOTE: 

* FSC is the abbreviation for FRC-Splitting-Control register. 

The setting of the Passive bit of t h e FRC -Splitting-Control prevents the passive component from 
driving the AP-bus or the BERI^-BERI^ lines. Since the passive component responds to I AC 
requests addressed to it, software can reunite a split pair. 
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For FRC splitting to function properly, duplication of the AP-bus arbitration lines is not allowed. 
Both the master and checker must be connected to the same set of arbitration lines, so that either the 
master or the checker can act as the sole active component for the module. If the arbitration lines were 
duplicated, then the two sets of arbitration lines would always be different after an FRC split. 

SOFTWARE CONSIDERATIONS 

Software performs two key recovery management functions: establishing the reconfiguration policy, 
and reviewing the recovery decisions made by the hardware. When a permanent error occurs, the 
hardware automatically removes the faulty module or bus in a QMR system. Software, however, can 
decide if the configuration is optimal, given the reduced resources available to the system. 

For example, consider the system shown in Figure 13-9. Assume that the primary processor module 
failed. The system hardware detects the error, reports the error, and replaces the primary unit with 
the shadow unit. The processor can be notified of the error condition if a BERL or a LERL line is 
connected to one of its interrupt input lines. The resulting system configuration can detect an error, 
but it cannot recover from a failure if a failure should occur in this module without a partner. Software 
needs to decide whether this module is to be remarried or be allowed to operate without a partner. 



PRIMARY 
PROCESSOR 




AP-BUS 



NOTE: 




FRC PAIR 
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Figure 13-9: System Vulnerability After Reconfiguration 

Software must also respond to transient errors by sending a Terminate-Permanent-Error-Window 
command after notification of this error by a processor. If the same transient error occurs twice in 
a row before the Terminate-Permanent-Error-Window command is issued, the transient error is 
classified as a permanent error (see the "Permanent Error Decision" for more details). 
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SUMMARY 

The 80960 hardware system recovers from transient errors by using a retry sequence. The retry 
sequence provides recovery for most frequent failure modes (RAM soft errors, noise on the 
backplane, etc.). If the retry sequence fails, then it is possible to use a redundant resource to mask 
the fault from the rest of the system. Resource reconfiguration can successfully mask any single 
failure in the system. Resource reconfiguration requires redundant resources to use the recovery 
mechanisms, such as module shadowing, bus switching, or FRC splitting. The entire recovery 
algorithm is implemented in the BXU, and is orthogonal to both detection and redundancy 
mechanisms. 
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CHAPTER 14 
INITIALIZATION 

This chapter describes the hardware requirements for initializing the 80960MC processor and the 
BXU in a fault-tolerant system design. The basic minimum initialization requirements are described, 
as well as some of the available options. The exact initialization procedure depends on the type of 
system configuration. For example, the system can be configured to support multiprocessors, 
various levels of fault tolerance, up to four AP-buses, up to 20 modules, etc. 

This chapter focuses on the initialization of a system that uses the BXU in multiprocessor and fault- 
tolerant configurations. There are at least two modules and one AP-bus interconnected with BXUs. 
The hardware initialization requirements for a single 80960MC processor configuration without 
BXUs is explained in Chapter 3. 

BXU INITIALIZATION 

This section describes how to initialize the BXUs in the system. Initialization of the BXU is divided 
into three phases: 

1 . During the default initialization phase the default values are established in the BXU registers, 
the BXUs are synchronized to the clock phase, and the local bus identification parameters are 
established. 

2. During the identification phase, the physical identification parameters of the BXU are 
established, and the board characteristics (type of board, amount of memory , etc.) may be loaded 
into the BXU. 

3 . During parameterization phase, the registers of the BXUs are loaded with any parameters that 
differ from the default values. During this phase, the logical identification parameters are set 
in the appropriate registers. 

The default initialization and identification phase is always required, regardless of the system 
configuration. The parameterization phase is optional and requires a 80960MC processor or another 
agent to perform the operations. 

Default Initialization Phase 

When the RESET signal is asserted, the BXU responds with the following actions: 

• Forcing all internal state machines to the idle state. 

• Synchronizing its clock phase to the global system clock phase. 

• Loading its registers with default values. 
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Clock Phase Synchronization and RESET Timing 

The assertion of RESET synchronizes the BXUs to a global system clock phase. Figure 14-1 
illustrates the timing of RESET. RESET is asserted relative to any rising edge of CLK2, and held 
asserted for at least 40 CLK2 cycles. RESET then is deasserted after the rising edges of CLK and 
CLK2, but before the next rising edge of CLK2. This establishes clock edge A. 

Because the fault-tolerant logic operates at half the AP-bus clock speed, the BXUs must be 
synchronized not only to the A edge, but to every other A edge. This implies that in a system operating 
at 1 6 MHz, the component clock is 32 MHz and the internal synchronizing clock for RESET is 8MHz 
(shown as CLK0.5 in Figure 14-1). 
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Figure 14-1 : RESET Timing 

The initialization parameters must be setup prior to the rising edge of CLK2, as shown in 14-1, and 
must be held beyond the first C edge. See the M82965 data sheet for the setup and hold time 
specifications. 

The outputs are placed in high impedance or are not asserted (for open-drain outputs) starting at the 
CLK2 rising edge at which RESET (asserted) is sampled. 
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All registers of the BXU except the Error-Record and the FRC -Splitting-Control registers are loaded 
in the same manner. The register bits are either loaded with the default value or the value sampled 
on an external set of pins when RESET is deasserted. 

The contents of the Error-Record register are not modified by the initialization sequence. The 
information in this register may be useful in determining the cause of a system crash. Thus, it is 
important for software to write to this register before reading it, unless software knows that the last 
RESET cycle was a warm RESET (see the "Cold and Warm Reset Distinction" section for definition 
of a warm RESET). This restriction is required to prevent FRC errors from occurring because the 
registers in the master and checker may have different values. 

The BXU loads the FRC -Splitting-Control register conditionally. A cold RESET signal clears the 
control bits of this register; a warm RESET signal does not modify any of the control bits. The FRC- 
Splitting-Control register is only loaded after cold RESET signal. 

Pins Sampled During Default Initialization 

Four of the BXU pins are sampled while RESET is asserted. This sampling occurs continually on 
the falling edge of the first phase of the internal clock. When the RESET signal is deasserted, the 
BXU uses the last values sampled for initialization. 

The chosen default values allow the BXU to function in a simple system configuration without 
requiring any external logic devices. The default values of each register are listed in Appendix A. 
The AP-bus and L-bus parameter settings are described in the following paragraphs. 

AP-Bus Parameters 



If the COM pin is high during RESET, then the B XU is a bus master in a master/checker FRC pair. 
The master drives the AP-bus, and the checker (COM low) monitors the master and reports any 
disagreement between the two 80960MC processors. 

L-Bus and Module Parameters 

IAC requests on the L-bus use the contents of the System-Bus-ID register for the L-Bus destination 
field (see Appendix A for the description of the System-Bus-ID register). The System-Bus-ID register 
assigns a unique identification to each BXU in the same module. The contents of this register are 
also used to identify the AP-bus for the BXU. The System-Bus-ID register is read from the MODCHK 
an d BOUT lin es du ring RE SET. Table 14-1 shows the identification assignment for different values 
of MODCHK and BOUT. 



The LERL pin is used to select the mode of operation on the BXU. If the LERL 1 pin is high, then the 
BXU-Mode bit in the LBI-Control register is set to a value of one to select Processor mode. 
Otherwise, the BXU operates in Memory mode. 
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Table 14-1: System-Bus-Identification Assignments 


AP-Bus 


System-Bus-ID-Register 






MODCHK* 


BOUT* 


BIT 15 


BIT 14 











H 


H 


1 





1 


H 


L 


2 


1 





L 


H 


3 


1 


1 


L 


L 



NOTE: 

*L (Low) indicates the line is asserted and H (High) indicates the line is deasserted. 

Loading Parameters 

The parameters must be valid four full bus cycles (eight CLK2 cycles) prior to the first CLK2 rising 
edge after RESET is deasserted. To accommodate relatively slow external controllers, the parame- 
ters may remain asserted on the pins until bus traffic begins on either the L-bus or the AP-bus of the 
BXU. Thus, the module with the fastest RESET logic must not generate AP-bus traffic until the 
slowest module has released its RESET parameters. In general, this condition is met when the 
80960MC processor performs the self-test function during initialization. 

The BXU disables the error reporting operation when it recei ves a RESET s ignal. This action 
preve nts a ny reac t ion bas ed on the parameters loaded on BOUT and MODCHK. Furthermore, the 
COM and LERLj-LERL lines are negative edge sensitive signals. Releasing the parameter can only 
cause a positive going edge. Thus, no reaction occurs in the BXU. 

BXU State at the End of RESET 

At the end of RESET each BXU in the module has a unique value in the System-Bus-ID register and 
has the I AC recognition function enabled. A 80960MC processor on the L-bus can use IAC type 
0000 B (Request to BXUs on L-bus) to address the BXU on the message bus. In this way, the 80960MC 
processor can access more information about the board configuration by using the serial protocol 
with the COM register. Note that until the board configuration is known, the 80960MC processor can 
only address the message BXU (the BXU on AP-bus ), because it is the only BXU guaranteed to be 
present. 

After the board configuration is known, the 80960MC processor can access all the BXUs in the 
module by IAC register requests using logical or physical addresses. The IAC 's module address must 
be zero because the Physical-ID and Logical-ID registers are at their default value. Therefore, these 
IAC requests are handled entirely within the module and will not generate any AP-bus traffic. 
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The INIT-RAM memory recognizer is enabled because the Disable-INIT-RAM bit in the LBI - 
Control register is zero (default value). This recognizer matches any memory request that has AD 31 
asserted and AD 30 deasserted. Addresses in this space are mapped into the external SRAM normally 
used by the cache. Thus, the lower half of memory can be made available for PROM storage and the 
upper-addresses can be reserved for memory used during initialization. The INIT-RAM memory 
recognizer can be disabled by setting the Disable-INIT-RAM bit in the LBI-Control register, if this 
function is not desired. 

Requests that match the INIT-RAM memory recognizer are handled by the loc al SRAM, and none 
of these requests flow onto the AP-bus. The BXU sequences the READY and WDj-WDq signals, 
and maps the LAD 15 and LAD 16 lines to the WY and WYj lines, respectively. This provides up to 
128K bytes of RAM storage (with four AP-buses) for use by initialization software. The signal 
sequencing uses the slow-read, slow-write cache timing mode (specified by the Cache-Configura- 
tion register). 

During RESET, the BXU enables the IAC address recognizers for the AP-bus and disables the 
memory address recognizer. In addition, the BXU disables the cache and prefetch units and marks 
the entire cache directory invalid. The BXU does not initiate any requests on either bus. It only 
responds to requests that it receives. 

The BXU does not send error reports, but it will correctly respond to any error reports that it receives. 
All BXUs have the correct Confinement-ID field in the Bus-Error-ID register. The Confinement- 
ID field in their Module-Err or -ID register contains FF H . The Error-Reporting-Enable bit in the FT I 
register and the Bus-Switch-Disable bit in the AP -Control register are cleared. Before operating on 
the AP-bus and using the error reporting network, the software should load information into two 
registers: the ID -Parity bit and the Source-ID field in the Bus-Error-ID register, and a unique 
Confinement-ID field in the Module-Err or-ID register. 

Identification Phase 

During this phase of initialization, the logical, physical and arbitration IDs for the BXUs are assigned 
by the Initialization processor. 

After default initialization is complete, the Identify Device Order IAC (IAC type Oil 1 B ) is used to 
load AP-bus identification information into the BXUs. Se nding an Identify Device Order causes a 
two- word write request to be generated on the AP-bus. If the INITID pin of a BXU is asserted during 
the first data cycle of this command, it accepts the Identify Device Order command. This special 
command does not cause the BXU to send a reply packet. 



The INITID pin of a BXU is wired to one (and only one) of the AP-bus address/data lines, as shown 
in Figure 14-2. The par ticular A /D line used is an arbitrary decision. For BXUs functioning as a 
master/checker pair, the INITID pins are tied together. 
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Figure 14-2: How the Identify Device Order Specifies a Particular BXU 

An encoded value is loaded from the data pins on the second data cycle to various registers in the 
BXU. Figure 14-3 shows the fields in the data words associated with this command. The ///// field 
of the second data word is loaded into the Component-ID field of the Physical-ID register and the 
Unit-ID field of the Logical-ID register. The DD field is loaded into the Drive field of the 
Arbitration-ID register, and the CCCC field is loaded into the Count field of the Arbitration-ID 
register (see Appendix A for the complete description of these registers). 
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Figure 14-3: Data Field for the Identify Device Order 

The use of the Identify Device order is restricted to special situations because of its special address 
recognition function. A BXU that receives this command must either be idle or processing another 
Identify Device Order. No other IAC or memory requests may be present in the BXU while it handles 
the Identify Device Order. 

This restriction is necessary because there may be more than one Initialization processor, each 
responsible for a portion of the total system initialization task. The identification phase of 
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initialization must be completed by all Initialization processors before they begin the parameteriza- 
tion phase of initialization. 

After receiving an Identify Device Order IAC, a BXU (or FRC pair of BXUs) has the same ID in 
the Logical-ID register and the Physical-ID register. The parameterization phase of initialization can 
be used to change the Unit-ID field of the Logical-ID register. These identification fields are used 
to address the BXUs during normal system operation. 

The Arbitration-ID register is used during normal operation to indicate a unique time when a BXU 
drives a request on the AP-bus. Although particular assignments can be any value between and 3 1 , 
the most efficient value assignments are packed, e.g., 00 H , 10 H , 20 H , 01 H , 1 1 H , 21 H , ..., 0F H , 1F H , 2F H . 
Duplicate values in the Arbitration-ID registers are not allowed because they will cause AP-bus 
contention and loss of pipeline ordering. 

Parameterization Phase 

The parameterization phase of initialization is only required if the BXU registers must be set to values 
other than the default values. During this phase of initialization, the values that were set up during 
the identification phase may be altered and numerous additional registers may be changed. 

Parameterization consists of two separate activities. During th e first activity, external information 
is sent to the BXU. This occurs during the period in which the COM pin is defined to function as a 
serial input to the COM register (see Appendix A for the complete description of the COM register). 
The COM register can be loaded by an external controller with specific board parameters. (The 
contents of the COM register are strictly arbitrary and it is not necessary to use this register for the 
parameterization phase if software uses other techniques to change the contents of the BXU 
registers.) 

For example, a controller on a memory board could load the COM register with a value that indicates 
that this board is a memory board with a certain amount of memory, type, etc. On the other hand, a 
controller on a processor board could load the COM register of its BXUs with a code that specifies 
that this board is a processor board, perhaps with a cache, etc. The protocol for loading the COM 
register is explained later in this section. 

During the second activity in the parameterization phase, the default values are changed consistent 
to the parameters that were loaded into the COM register. The BXU itself does not use the contents 
of the COM register to change its default values. Software must read the COM register and determine 
how to change the registers of the BXU with the new parameters. 

Serial COM Protocol 

The protocol begins with a start transition, and loads up to 32 bits of information into the COM 
register. The serial loading of this register depends on a multiple-sample protocol, where the sample 
points are defined by the BXU. The COM protocol for loading a single bit of information from the 
COM pin is illustrated in Figure 14-4. 



14-7 



iny 



INITIALIZATION 



START CONDITION - 

-s 2 — h* — s 3 — 




y yy - : ri , \ V , V i -^ 



271079-185 



Figure 14-4: COM Pin Protocol 



The sequence of events required to load the COM register are listed below. 



1 . The start condition is detected by the high-to-low transition of the COM signal. The BXU detects 
the start condition when the COM pin is high for at least two clock cycles (S { and S 2 ), and then 
the COM is low for at least two clock cycles (S 3 and S 4 ). Neither a high nor a low level signal 
triggers the COM protocol. 

When the start condition is detected, the following occurs: 



2. 



a. 
b. 
c. 



d. 



The Test-Enable bit in the Test-Detection register is cleared. 
The Com-Register-Altered bit in the AP -Control register is set. 



An COM-Altered error report is sent by the BXU on the BERL r BERL lines if the Corn- 
Reporting bit is set in the FT1 register. This occurs only if the Com-Register-Altered bit in 
the AP -Control register was not set prior to the start condition detection. 



Other usages of the COM register are disabled. The usages of the COM pin and COM 
register include the following: 

i. Testing the BXU's FRC logic (see Chapter 12). 



Signalling permanent errors: the BXU drives the COM pin when the Drive-Corn bit in 
the AP -Control register is set, or when the Faulty bit in the FT2 register is set. 



e. On the twenty-eighth clock cycle following S 4 , the value on the COM pin is latched. Note 
that the COM pin may go high at any time following S 5 , and the COM pin may go low at 
any time following S 2 for a new start. 

3. For each subsequent start condition, the sampled value is shifted into the COM shift chain. 



The latched value is shifted into the COM shift chain, as illustrated in Figure 14-5. Note that the bits 
are shifted in a special sequence. The first bit shifted in appears in bit 16 of the COM register. The 
last bit shifted in appears in bit 15 A total of 38 bits of data must be shifted in, 32 bits are visible and 
six bits are hidden. This means that software reads 32 bits (bit 31 through bit ) from the COM register; 
the hidden bits are not read. 
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Figure 14-5: COM Register Loading by Using The COM Pin 

Write operations to this COM register clear the hidden bits. The hidden bits are part of a shift chain 
used for testing the FRC logic of the BXU. See Chapter 12, the "FRC Logic" section, for more 
information on testing the FRC logic. 

PROGRAMMING NOTE 

A potential race condition exists between software and hardware usage of the COM register. The software must 
wait for the serial data transfer to complete before it reads data from the COM register. One way to detect the 
completion of loading 38 bits is to compare the contents of the COM register with the previous contents of the 
COM register, and not clear the Corn-Register- Alteredbit When there is no change in the data, then the transfer 
is complete. Another method would be to employ a microcontroller that sets a hardware flag when the transfer 
is completed. 

Every time the COM-Register-Altered bit is set, a COM-Altered error report is generated by the BXU if the 
COM-Reporting bit in the FT1 register is set. The COM protocol makes no provision for detecting the end 
of the loading sequence. 

Cold and Warm Reset Distinction 

The BXU is capable of distinguishing between a power-up RESET (cold RESET) and a RESET that 
comes during normal operation ( warm RESET). The BXU uses the distinction between a warm and 
cold RESET to control the FRC splitting logic. FRC splitting is an option that allows an FRC pair 
to be split under software control. An FRC split occurs only on warm RESET. 

The V REF pin of the BXU is used to distinguish between a warm or cold RESET. To detect a cold 
RESET, an external resistor/capacitor circuit on the V REF pin can be designed to pull the signal above 
V REF/C (called the cold trip point) at the time of power-up, as shown in Figure 14-6. The values of 
the resistor/capacitor network must be chosen such that the V REF pin is held above the CO/d trip point 
until the RESET signal reaches a stable voltage level (either high or low). After RESET reaches a 
stable voltage, the V REF pin is brought to V REF/W . RESET must be stable during the transition of V REF 
from the CO/d trip point to V REF/W , as shown in Figure 14-7. The first high-to-low transition of RESET 
after V REF drops below the cold trip point is considered a CO/d start. Assertions of RESET after the 
first high-to-low transition are considered warm RESETs. All outputs of the BXU float to a high 
impedance state on the leading edge of RESET. RESET can be asserted at any time. 

If V REF is brought above the maximum normal operating value (V REF/W Max ) again, it must be followed 
by a RESET sequence. The first assertion of RESET after the V REF pin has reached the trip point is 
labeled a cold RESET. All subsequent RESET signals are considered warm RESETs. After the 
leading edge of every RESET pulse, the Warm-RESET bit in the FRC -Splitting-Control register 
indicates whether the initialization was warm or cold RESET. 
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Figure 14-6: RC Network for V RI 
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Figure 14-7: Relationship Between V REF and RESET 
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PROGRAMMING NOTE 

V REF must be brought to V RE in order to reset the FRC -Splitting-Control register, otherwise it will be loaded 
with random values. 

MODULE SHADOWING 

Primary/shadow FRC pairs can be married during initialization or during system operation using 
another module or a special agent. Chapter 1 3 described how to marry these pairs during system 
operation. The following paragraphs show how to marry a primary /shadow pair during initialization 
using a special agent. 

Marrying a Primary/Shadow Pair Using a Special Agent 

Figure 14-8 shows the configuration that marries a primary/shadow pair during initialization using 
a special agent. During RESET, the master 80960 MC proce ssor in the Primary-Electxmit is setup 
as the initialization processor (a high voltage on the BAD AC pin). Thus, the Primary-Elect Master 
performs all of the necessary register configuration for all the BXUs in the (future) QMR module. 
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Figure 14-8: Marrying a Primary/Shadow Pair Using a Special Agent 

After completing the register configuration, the processor in the Primary- Elect signals the special 
agent to send a "Restart Processor" IAC using the logical address of the primary/shadow pair (see 
the 80960MC Programmer' s Reference Manual for more details on the different types of message 
IAC requests). This special agent delays sending the message IAC to allow enough time for the 
initialization processor in the Primary- Elect unit to send a "Stop-Processor" IAC to itself causing 
it to enter the stopped state. 

The following paragraphs enumerate the steps that the initialization processor performs to configure 
the primary/shadow pair for marriage. These steps follow the normal identification phase of the BXU 
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initialization. If there are multiple buses in the system, any action performed on one BXU in a module 
is performed on all BXUs in that module. 

1 . With the "Register Request Using a Physical Address" IAC (type 0100 B ), set the Shadow bit in 
the QMR register of the shadow BXUs. 

2. Copy the AP-Mask and AP -Match registers from the BXUs in the Primary-Elect unit to the 
BXUs in the Shadow- Elect unit 

3. With the "Register Request Using a Physical Address" IAC (type 0100 B ), prepare the shadow 
for marriage by performing the following actions: 

a. Copy the contents of the registers in the Primary-Electunit to the registers of the Shadow- 
Elect unit except for the QMR, Spouse-ID, Module-Error-ID, Bus-Error-ID, Logical-ID, 
and Physical-ID registers. 

Because the Arbitration-ID register is write-only register, software must write to both 
registers with the same value. 

b. Set the Module-Error-ID register in the Shadow-Elect unit to a unique value. 

c. Set up the Bus-Error-ID register in the Shadow- Elect unit by setting the Source-ID to a 
unique value. 

4. With the "Register Request Using a Physical Address" IAC (type 0100 B ), make the units aware 
of each other by performing the following actions: 

a. Copy the Confinement-ID field of the Module-Error-ID register of the Primary-Electunit 
to the Spouse-ID register of the Shadow- Elect unit. 

b. Copy the Confinement-ID field of the Module-Error-ID register of the Shadow-Electunit 
to the Spouse-ID register of the Primary-Elect unit. 

5. With the "Register Request Using a Physical Address" IAC (type 0100 B ), copy the Logical-ID 
number of the Primary-Elect unit to the Shadow-Elect unit. 

6. Marry the Primary-Elect/Shadow-Elect unit, using IAC type 1 00 B (Register Request Using 
a Physical Address). Marriage is accomplished by setting the Married bit in the QMR register 
of the Shadow-Elect unit and Primary-Electunit. This marks the units as married. 

7. With IAC type 0100 B (Register Request Using a Physical Address), set up the married units to 
recognize the same memory ranges. Address recognition is accomplished by performing the 
following functions: 

a. Set the AP -Match registers to the desired value. 

b. Set the AP-Mask registers to the desired value. 

c. Set the Enable bit in the AP -Match registers. 

Special Agent Overview 

After completing the register configuration, the processor in the Primary-Elect signals the special 
agent to send a "Restart Processor" message IAC using the logical address of the primary/shadow 
pair. This special agent delays sending the "Restart Processor" IAC to allow enough time for the 
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processor in the Primary-Elect unit to send a "Stop-Processor" I AC to itself* causing it to enter the 
stopped state. '■•-■> - 

Figure 14-9 illustrates one way in which to implement this special agent. The logic sends a "Restart- 
Processor" IAC message after a delay of approximately 20 us. The logic can be attached to the L- 
bus of a any BXU in Processor mode (the BXU has to be in Processor mode in order to drive the AP- 
bus). ...■.' '' ■ ""■'•'' - ; 
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Figure 14-9: Special Agent Logic 

Figure 14-10 shows the address and data words of the IAC message that is stored in memory. The 
first word access contains the IAC message address. The second word contains the "Restart-Proc- 
essor" IAC message. The third and fourth words contain the segment table addfess and Processor 
Control Block (PRCB) address, respectively (see the 80960MC ProgrWmef s Reference Manual 
for more information on this type of IAC message). The "Message" IAC (type 001 1 B ) shown in 
Figure 14-1 1 shows the address format for the Restart IAC that must reside in the Special Agents 
RAM. 

The IAC generator described in Chapter 9 can be adapted to include the special agent logic by making 
two modifications: the addition of delay times to to allow the processor enough time to send a "Stop- 
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Processor" IAGLtoltseiCafld the capability to generate a "Restart Processor" IAC. The modified 
configuration is shown in Figure 14-12. 
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Figure 14-10: Contents of the "Restart-Processor" IAC 
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Figure 4-11 : Address Format for the Restart IAC 
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Figure 14-12: IAC Generation Logic 
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The function of the logic blocks are identical to the configuration explained in Chapter 9. The address 
decoder, however, also responds to a memory-mapped address that results in sending a "Restart 
Processor" IAC. The state machine inserts delay times to ensure that the processor has adequate time 
to send the "Stop-Processor" IAC to itself. Figure 14-13 shows the additional states (noted by the 
shaded areas) required to perform this task. The Restart signal indicates that the processor requests 
a "Restart Processor" IAC. Table 14-2 defines each state including the new states. Note that the 
Counter Wait state (CW) waits for the counter to time-out, then the state machine sequences through 
IAC Message state zero (IM ), IAC Message state one (IMj), IAC Message state two (IM 2 ), IAC 
Message state three (IM 3 ), and the Check state (CK). 
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Figure 14-13: Modified State Diagram for IAC Generator 
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Table 14-2: Modified State Functions 


State 


Name 


Signals 
Asserted 


Other Action 


Comments 


I 


Ideal 


None 


Set MASK bit after 
reset 


Waiting for INT from M8259A, 
or HOLDR from BXU and place 
READY in high impedance 


A o 


AddresSo 


HLDA 


Latch LAD 4 -LAD q if 
ASD is asserted 


Grant bus to BXU ; place ADS 
in high impedance and drive 
READY (to deasserted state) 


^ 


Address, 


HLDA 


Decode latched 
address 


Wait here if SEL not asserted 


DAo 


Data Accesso 


HLDA, RD, 
orWR 


Start delay timer 


Wait for DLY to meet M8259A 
TRLRH spec 


DA, 


Data Access, 


HLDA, RD, 
orWR, 
READY 


Set or reset MASK 
bit if commanded 


Latch data at end of cycle if 
write operation; assert 
READY to complete transfer 


BW 


Bus Wait 


HLDA 


None 


Wait for BXU to release L-Bus 


CW 


Counter Wait 


None 


None 


Wait for 400 clock cycles to 
time-out 


IM 


IAC Message 
Address 


ADS.RD 


Increment IAC RAM 

address at end of 

cycle 


Send IAC address 


IM, 


IAC Message 
Data Word, 


RD 


Increment IAC RAM 

address at end of 

cycle 


Send Data Word,; increment 
address Modulo 2 if no restart, 
otherwise Modulo 4 


IM 2 


IAC Message 
Data Word 2 


RD 


Increment IAC RAM 

address at end of 

cycle 


Send Data Word 2 ; increment 
address Modulo 4 


IM 3 


IAC Message 
Data Word 3 


RD 


None 


Send Data Word 3 


CK 


Check for 
BAbAC 


None 


None 




If BADAC asserted retry IAC 
message 


IA 


Interrupt 
Acknowledge 


INtA 


None 


Wait for SLY to meet M8259A 
TRLRH spec 


II 


Interrupt Idle 
Time 


None 


None 


Wait for DLY to meet M8259A 
TRHRL spec 
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Table 14-2: Modified State Functions (cont.J 



State 


Name 


Signals 
Asserted 


Other Action 


Comments 


VF 


Interrupt 
Vector Fetcho 


INTA 


*Latch shifted 

vector at end of 

cycle 


Wait for DLY to meet M8259A 
TRLRH spec 


VF, 


Interrupt 
Vector Fetch! 


INTA 


None 


Interrupt vector decode 
wait 



NOTE: 

* Since each interrupt I AC message occupies two words in RAM, the vector must be shifted left by 1 bit in 
order to serve as a valid IAC message address. 

QMR INITIALIZATION EXAMPLE 

This section illustrates the initialization process with a step-by-step example. The system configu- 
ration consists of a processor primary/shadow pair and an I/O primary/shadow pair, as shown in 
Figure 14-14. In this example, there is no cache and no global memory. The I/O module contains 
memory that is accessed periodically by monitor sensors. The I/O module also has logic to generate 
a "Restart Processor" message IAC, which is triggered by a write operation to a particular location 
in the I/O space. There are two buses in the system where AP-buSj is a passive backup (i.e., no 
interleaving is done). 
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Figure 14-14: Configuration for Example 

The names for the eight BXU pairs in the example configuration are shown in the following list: 

• PP identifies the BXU pair in the processor Primary-Elect unit on AP-bus 

• PPj identifies the BXU pair in the processor Primary-Elect unit on AP-buSj 

• PS identifies the BXU pair in the processor Shadow-Elect unit on AP-bus 
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• PS, identifies the BXU pair in the processor Shadow-Elect unit on AP-bus, 

• I/OP identifies the BXU pair in the I/O Primary-Elect unit on AP-bus 

• I/OP, identifies the BXU pair in the I/O Primary-Elect unit on AP-bus, 

• I/OS identifies the BXU pair in the I/O Shadow-Elect unit on AP-bus 

• I/OS , identifies the BXU pair in the I/O Shadow-Elect unit on AP-bus , . 

The following sections show the appropriate address and data words used to initialize each BXU in 
this example system configuration. The description first examines the parameterization and 
identification phases, then describes the marriage sequence. This example follows the step-by-step 
guidelines presented in the "Module Shadowing" section for the marriage sequence. An 80960MC 
processor in the Primary-Elect unit performs the initialization actions. 

Local BXU Parameterization 



The initialization is performed on the BXUs residing on the initialization processors L-Bus prior to 
the identification phase to establish the error reporting network. Then all registers that need to be 
changed from their RESET default values in the remaining BXUs are initialized except for the QMR 
register. The QMR register is initialized as part of the marriage sequence. 

The following list of addresses, data, and actions illustrates the parameterization phase for the BXUs 



Action 



Single- word write operation to the LBl-Control register of both 
BXUs using the "Register Request from the L-bus" IAC (IAC 
access type 0000 B ). This operation sets the BXU in Processor 
mode as a secondary bus master and disables the INIT-RAM 
address recognizer. 



PP and PP,. 




Address 


Data 


(Hex) 


(Hex) 


FF000148* 


00000030 



FF000 1 64 FFFC0000 Single- word write operation to the Mask register of both BXUs 

using the "Register Request from the L-bus" I AC (I AC access type 
0000 B ). This operation sets the appropriate bits in the Mask 
register. 

FF000 1 60 0004000 1 Single- word write operation to the Match register of both BXUs 

using the "Register Request from the L-bus" IAC (IAC access type 
0000 B ). This operation enables the bus recovery of the Mask and 
Match registers. 
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Address 
(Hex) 



Data 
(Hex) 



Action 



FF000240 00000004 Single-word write operation to the Prefetch-Control register tf 

both BXUs using the "Register Request from the L-bus" IAC (I AC 
access type 0000 B ). This operation makes both prefetch channels 
available. 

FF000054 00000007 Single-word write operation to the FT1 register of both BXUs 

using the "Register Request from the L-bus" IAC (IAC access type 
0000 B ). This operation selects the Wait-for-BERL timing, enables 
the error reporting network, and ensures proper fault-tolerant 
operation. 

FF0010E4 00000401 Single- word write operation to the Module-Error-ID register of 

the BXU on AP-bus using the "Register Request (Physical Ad- 
dress)" IAC (IAC access type 0100 B ). This operation sets the Con- 
finement-ID field to a value of four and the Source-ID field to a 
value of zero. 

FF0050E4 00000402 Single-word write operation to the Module-Error-ID register of 

the BXU on AP-busl using the "Register Request (Physical 
Address)" IAC (IAC access type 0100 B ). This operation sets the 
Confinement-ID field to a value of four the Source-ID field to a 
value of one. 

FF0008E8 00000009 Single- word write operation to the Bus-Error-ID register of the 

BXU on AP-bus using the "Register Request (Physical Address)" 
IAC (IAC access type 0100 B ). This operation sets the Confine- 
ment-ID field to a value of zero the Source-ID field to a value of 
four. 

FF0048E8 00000108 Single-word write operation to the Bus-Error-ID register of the 

BXU on AP-buSj using the "Register Request from the L-bus" IAC 
(IAC access type 0000 B ). This operation sets the Confinement-ID 
field to a value of one the Source-ID field to a value of four. 

FF0000C4 00000500 Single-word write operation to the Spouse-ID register of both 

BXUs using the "Register Request from the L-bus" IAC (IAC 
access type 0000 B ). This operation sets the Spouse-ID field to a 
value of five. 
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*Note: The following sample of assembly code can be used for the first write operation. All other 
write operations can use the same sequence of instructions. 



Instruction 


Address/Data 


LDCONST 


0xFF000148,G ( 


LDA 


IAC DATA, G, 


SYNMOV 


G ,G, x 


BE 


ERROR 



Comment 

Load the address in register G 

Load the data in register G { 

Move data to the address contained in register G 

Report an error if it occurs 



IAC DATA: 



WORD Ox 00000030 



Identification Phase and Primary/Shadow Marriage 

The identification phase is divided into two sections: the assignment of identification values for each 
BXU and the initialization of the I/O primary/shadow units. These topics and the marriage of the 
processor primary/shadow units are described in this section. 

BXU Identification Assignment 

During this phase, six BXU pairs are assigned logical, physical, and arbitration identification values 
by using the "Identify Device Order" IAC. The primary processor BXUs on AP-bus and AP-buSj 
use the default values of the appropriate registers. Consequently these BXUs are not assigned new 
values. 



Address 
(Hex) 



Data 
(Hex) 



Action 



FF001C00 00000002 Two-word write operation to the Logical-ID, Physical-ID, andAr- 

00040000 bitration-ID registers of the PS BXU using the "Identify Device 
Order" IAC (IAC access type 01 1 1 B ). The Component-ID field of 
the Physical-ID register is set to a value of 1 H and the Drive and 
Count fields of the Arbitration-ID register are set to a value of zero. 
The Unit-ID field of the Logical-ID register is also set to a value 
of 01 H , but this value is changed later. 

FF001C00 00000004 Two-word write operation to the Logical-ID, Physical-ID, and Ar 

00080001 bitration-ID registers of the I/OP BXU using the "Identify Device 
Order" IAC (IAC access type 01 1 1 B ). The Component-ID field of 
the Physical-ID register is set to a value of 02 H , the Drive field of 
the Arbitration-ID register to a value of zero, and the Count field 
of the Arbitration-ID register to a value of one. The Unit-ID field 
of the Logical-ID register is also set to a value of 02 H , but this value 
is changed later. 
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Address 
(Hex) 



Data 
(Hex) 



Action 



FFOO 1 COO 00000008 Two-word write operation to the Logical-ID, Physical-ID, and Ar- 

000C000 1 bitration-ID registers of the I/OS BXU using the "Identify Device 
Order" IAC (IAC access type 01 1 1 B ). The Component-ID field of 
the Physical-ID register is set to a value of 03 H , the Drive field of 
the Arbitration-ID register to a value of zero, and the Count field 
of the Arbitration-ID register to a value of one. The Unit-ID field 
of the Logical-ID register is also set to a value of 03 H , but this value 
is changed later. 

FF005C00 00000002 Two-word write operation to the Logical-ID, Physical-ID, and Ar- 

00040000 bitration-ID registers of the PS, BXU using the "Identify Device 
Order" IAC (IAC access type 01 1 1 B ). The Component-ID field of 
the Physical-ID register is set to a value of 01 H , the Drive field of 
the Arbitration-ID register to a value of zero, and the Count field 
of the Arbitration-ID register to a value of zero. The Unit-ID field 
of the Logical-ID register is also set to a value of 01 H , but this value 
is changed later. 

FF005C00 00000004 Two-word write operation to the Logical-ID, Physical-ID, and 

00080001 Arbitration-ID registers of the I/OP, BXU using the "Identify 
Device Order" IAC (IAC access type 01 1 1 B ). The Component-ID 
field of the Physical-ID register is set to a value of 02 H , the Drive 
field of the Arbitration-ID register to a value of zero, and the Count 
field of the Arbitration-ID register to a value of one. The Unit-ID 
field of the Logical-ID register is also set to a value of 02 H , but this 
value is changed later. 

FF005C00 00000008 Two- word write operation to the Logical-ID, Physical-ID, and Ar- 

000C000 1 bitration-ID registers of the I/OS , BXU using the "Identify Device 
Order" IAC (IAC access type 01 1 1 B ). The Component-ID field of 
the Physical-ID register is set to a value of 03 H , the Drive field of 
the Arbitration-ID register to a value of zero, and the Count field 
of the Arbitration-ID register to a value of one. The Unit-ID field 
of the Logical-ID register is also set to a value of 03 H , but this value 
is changed later. 

FF04 1 044 00000000 Single- word write operation to the Logical-ID register of the PS 

BXU employing the "Register Request Using a Physical Address" 
IAC (IAC access type 0100 B ). This operation sets the Unit-ID to 
a value of zero. 

FF045044 00000000 Single- word write operation to the Logical-ID register of the PS , 

BXU employing the "Register Request Using a Physical Address" 
IAC (IAC access type 0100 B ). This operation sets the Unit-ID field 
to a value of zero. 
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Initializing the BXUs in the I/O Primary/Shadow Units 

For the I/O primary and shadow units, all the registers of the BXUs that require change from their 
RESET default values are initialized at this time including the QMR registers. The I/O modules are 
married with the toggling enabled. This section presents the initialization steps for each BXU in the 
I/O Primary-Elect/Shadow-Elect units. 



Initializing the I/O Primary-Elect BXU on AP-Bus 

Action 



Address 
(Hex) 



Data 
(Hex) 



FF08 1 044 00040000 Single- word write operation to the Logical-ID register of the I/OP 

BXU employing the "Register Request Using a Physical Address" 
I AC (I AC access type 0100 B ). This operation sets the Unit-ID field 
to a value of one. 

FF0410E4 00000600 Single-word write operation to the Module-Error-ID register of 

the I/OP BXU employing the "Register Request Using a Physical 
Address" IAC (IAC access type 0100 B ). This operation sets the 
Confinement-ID field to a value of six and Source-ID field to a 
value of zero. 

FF08 10E8 0000000C Single- word write operation to the Bus-Error-ID register of the 

I/OP BXU employing the "Register Request Using a Physical Ad- 
dress" IAC (IAC access type 0100 B ). This operation sets the Con- 
finement-ID field to a value of zero and the Source-ID field to a 
value of six. 

FF08 1 0C4 00000700 Single- word write operation to the Spouse-ID register of the I/OP 

BXU employing the "Register Request Using a Physical Address" 
IAC (IAC access type 0100 B ). This operation sets the Spouse-ID 
field to a value of seven. 

FF08 10E0 0000000E Single-word write operation to the QMR register of the I/OP BXU 

employing the "Register Request Using a Physical Address" IAC 
(IAC access type 0100 B ). This operation sets Married, and Toggle- 
Primary I Shadow control bits. Note that the write enable for the 
Married bit is turned on. 
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Initializing the I/O Shadow-Elect BXU on AP-Bus 

Action 



Address 
(Hex) 



Data 
(Hex) 



FF0C1044 00040000 Single-word write operation to the Logical-ID register of the I/OS 

BXU employing the "Register Request Using a Physical Address" 
IAC (IAC access type 0100 B ). This operation sets the Unit-ID field 
to a value of one. 

FF0C10E4 00000701 Single- word write operation to the Mo dule-Error-ID register of 

the I/OS BXU employing the "Register Request Using a Physical 
Address" IAC (IAC access type 0100 B ). This operation sets the 
Confinement-ID field to a value of seven and the Source-ID field 
to a value of zero. 

FF0C 10E8 0000000F Single-word write operation to the Bus-Error-ID register of the 1/ 

OS BXU employing the "Register Request Using a Physical Ad- 
dress" IAC (IAC access type 0100 B ). This operation sets the Con- 
finement-ID field to a value of zero and the Source-ID field to a 
value of seven. 

FF0C 10C4 00000600 Single-word write operation to the Spouse-ID register of the I/OS 

BXU employing the "Register Request Using a Physical Address" 
IAC (IAC access type 0100 B ). This operation sets the Spouse-ID 
field to a value of six. 

FF0C 10E0 0000003E Single- word write operation to the QMR register of the I/OS BXU 

employing the "Register Request Using a Physical Address" IAC 
(IAC access type 1 00 B ). This operation sets the Married, Toggle- 
Primary I Shadow, and Shadow control bits. Note that the write 
enable for the Shadow and Married bits is turned on. 



14-23 



iny 



INITIALIZATION 



Initializing the I/O Primary-Elect and Shadow-Elect BXUs on AP-Bus ) 

The "Register Request Using a Logical Address" IAC can be used to write to both the I/O Primary- 
Elect and Shadow-Elect BXUs. Although this action saves some programming steps, it is dangerous 
to use "Register Request Using a Logical Address" IAC on the AP-bus before the Unit-ID field in 
the Logical-ID register is assigned in all BXUs. 



Address 
(Hex) 



Data 
(Hex) 



Action 



FF04086C FFFC0000 Single- word write operation to the A? -Mask register of the both 

BXUs employing the "Register Request Using a Logical Address" 
IAC (IAC access type 001 B ). This operation sets the address 
recognition range and configures the BXU for no interleaving. 

FF040868 0004000 1 Single- word write operation to the AP -Match register of the both 

BXUs employing the "Register Request Using a Logical Address" 
IAC (IAC access type 0010 B ). This operation partitions the 
address recognition range and configures the BXU for no inter- 
leaving. 

FF040948 000000F0 Single- word write operation to the LBI-Control register of the both 

BXUs employing the "Register Request Using a Logical Address" 
IAC (IAC access type 0010 B ). This operation sets the BXU that 
is in Processor mode as a primary bus master and disables the INIT- 
RAM memory recognizers. 

FF040854 00000007 Single-word write operation to the FT1 register of the both BXUs 

employing the "Register Request Using a Logical Address" IAC 
(IAC access type 0010 B ). This operation selects the Wait-for- 
BERL timing, enables the error reporting network, and ensures 
proper fault-tolerant operation. 
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Initializing the I/O Primary-Elect BXU on AP-Bus 1 



Address 
(Hex) 



Data 
(Hex) 



Action 



FF085044 00040000 Single- word write operation to the Logical-ID register of the I/OPj 

BXU employing the "Register Request Using a Physical Address" 
IAC (IAC access type 0100 B ). This operation sets the Unit-ID field 
to a value of one. 

FF0850E4 00000603 Single-word write operation to the Module-Err or -ID register of 

the I/OP 1 BXU employing the "Register Request Using a Physical 
Address" IAC (IAC access type 0100 B ). This operation sets the 
Confinement-ID field to a value of six and the Source-ID field to 
a value of one. 



FF0850E8 



0000010D 



FF0850C4 



00000700 



Single- word write operation to the Bus-Error-ID register of the 1/ 
OPj BXU employing the "Register Request Using a Physical Ad- 
dress" IAC (IAC access type 0100 B ). This operation sets the Con- 
finement-ID field to a value of one and the Source-ID field to a 
value of six. 

Single- word write operation to the Spouse-ID register of the I/OPj 
BXU employing the "Register Request Using a Physical Address" 
IAC (IAC access type 0100 B ). This operation sets the Spouse-ID 
field to a value of seven. 



FF0850E0 0000000E Single- word write operation to the QMR register of the I/OP t BXU 

employing the "Register Request Using a Physical Address" IAC 
(IAC access type 0100 B ). This operation sets the Married, and 
Toggle-Primary/ Shadow control bits. Note that the write enable 
for the Married bit is turned on. 
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Initializing the I/O Shadow-Elect BXU on AP-Bus 1 

Action 



Address 
(Hex) 



Data 
(Hex) 



FF0C5044 00040000 Single-word write operation to the Logical-ID register of the I/OS , 

BXU employing the "Register Request Using a Physical Address" 
IAC (IAC access type 0100 B ). This operation sets the Unit-ID field 
to a value of one. 

FF0C50E4 00000702 Single-word write operation to the Module-Error-ID register of 

the I/OS j BXU employing the "Register Request Using a Physical 
Address" IAC (IAC access type 0100 B ). This operation sets the 
Confinement-ID field to a value of seven and the Source-ID field 
to a value of one. 

FF0C50E8 0000010E Single- word write operation to the Bus-Error-ID register of the 1/ 

OS j BXU employing the "Register Request Using a Physical Ad- 
dress" IAC (IAC access type 0100 B ). This operation sets the Con- 
finement-ID field to & value of om and the Source-ID field to a. 
value of seven. 

FF0C50C4 00000600 Single-word write operation to the Spouse-ID register of the I/OSj 

BXU employing the "Register Request Using a Physical Address" 
IAC (IAC access type 0100 B ). This operation sets the Spouse-ID 
field to a value of six. 

FF0C50E0 0000003E Single-word write operation to the QMR register of the I/OSj BXU 

employing the "Register Request Using a Physical Address" IAC 
(IAC access type 1 00 B ). This operation sets the Married, Toggle- 
Primary! Shadow, and Shadow control bits. Note that the write 
enable for the Shadow and Married bits is turned on. 
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Initializing the I/O Primary-Elect and Shadow-Elect BXUs on AP-Bus r 

The "Register Request Using a Logical Address" IAC can be used to write to both the I/O Primary- 
Elect and Shadow-Elect BXUs on AP-bus r Although this action saves some programming steps, 
it is dangerous to use "Register Request Using a Logical Address" IAC on the AP-bus before the Unit- 
ID field in the Logical-ID register is assigned in all BXUs. 



Address 
(Hex) 



Data 
(Hex) 



Action 



FF04486C FFFC0000 Single- word write operation to the AP-Mask register of the both 

BXUs employing the "Register Request Using a Logical Address" 
IAG (IAC access type 0010 B ). This operation sets the address 
recognition range and configures the BXU for no interleaving. 

FF044868 0004000 1 Single- word write operation to the AP -Match register of the both 

BXUs employing the "Register Request Using a Logical Address" 
IAC (IAC access type 0010 B ). This operation partitions the 
address recognition range and configures the BXU for no inter- 
leaving. 

FF044948 000000F0 Single-word write operation to the LBI-Control register of the both 

BXUs employing the "Register Request Using a Logical Address" 
IAC (IAC access type 00 10 B ). This operation sets the BXU that 
is in Processor mode as a primary bus master, and disables the 
INIT-RAM memory recognizers. 

FF044854 00000007 Single- word write operation to the FT1 register of the both BXUs 

employing the "Register Request Using a Logical Address" IAC 
(IAC access type 0010 B ). This operation selects the Wait-for- 
BERL timing, enables the error reporting network, and ensures 
proper fault-tolerant operation. 

Processor Primary-Elect and Shadow-Elect Marriage 

The BXUs in the I/O Primary and Shadow units are already married and awaiting commands from 
the 80960MC processor. The BXUs in the processor Primary and Shadow units are initialized 
except for the QMR register. The steps outlined in the "Module Shadowing" section of this chapter 
are followed in this example to complete the marriage of the processor Primary and Shadowunits. 
Each number in the "Step" column corresponds to the appropriate step outlined in the "Module 
Shadowing" section. 

After the marriage is completed, the 80960MC processor in the Primary unit sends a command to 
the special agent, which in turn, sends a "Restart Processor" IAC to the 80960MC processor in the 
Primary/Shadow modules. The pair enables toggling by performing a write operation to their 
respective registers using a "Register Request From the L-bus" IAC. 
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Step Address 
(Hex) 



Data 
(Hex) 



Action 



1. FF0410E0 



00000030 Single-word write operation to the QMR register of the PS 
BXU employing the "Register Request Using a Physical 
Address" IAC (IAC access type 0100 B ). This operation 
clears Toggle-Primary I Shadow] control bit and sets the 
Shadow bit. Note that write enable for the Shadow bit is 
turned on. 



FF0450E0 00000030 Single- word write operation to the QMR register of the PS , 

BXU employing the "Register Request Using a Physical 
Address" IAC (IAC access type 0100 B ). This operation 
clears the Toggle-Primary/Shadow control bit and sets the 
Shadow bit. Note that the write enable for the Shadow bit is 
turned on. 

2. No action required because the system configuration does 

not have global memory. 

3A. FF041 148 00000030 Single-word write operation to the LBI-Control register of 

the PS BXU employing the "Register Request Using a 
Physical Address" IAC (IAC access type 0100 B ). This 
operation sets the BXU that is in Processor mode as a 
secondary bus master and disables the INIT-RAM address 
recognizer. 

FF045 148 00000030 Single- word write operation to the LBI-Control register of 

the PS { BXU employing the "Register Request Using a 
Physical Address" IAC (IAC access type 0100 B ). This 
operation sets the BXU that is in Processor mode as a 
secondary bus master and disables the INIT-RAM address 
recognizer. 



FF041164 



FF045164 



FFFC0000 



FFFC0000 



Single-word write operation to the Mask register of the PS 
BXU employing the "Register Request Using a Physical 
Address" IAC (IAC access type 0100B). This operation sets 
the appropriate bits in the Mask register. 

Single- word write operation to the Mask register of the PS } 
BXU employing the "Register Request Using a Physical 
Address" IAC (IAC access type 0100 B ). This operation sets 
the appropriate bits in the Mask register. 
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Step Address 
(Hex) 



Data 
(Hex) 



Action 



FF04 1 1 60 0004000 1 Single- word write operation to the Match register of the PS 

BXU employing the "Register Request Using a Physical 
Address" IAC (IAC access type 0100 B ). This operation 
enables the bus recovery of the Mask and Match registers. 

FF045 1 60 0004000 1 Single- word write operation to the Match register of the PS , 

BXU employing the "Register Request Using a Physical 
Address" IAC (IAC access type 0100 B ). This operation 
enables the bus recovery of the Mask and Match registers. 

FF04 1 240 00000004 Single-word write operation to the Prefetch-Control regis- 

ter of the PS BXU employing the "Register Request Using 
a Physical Address" IAC (IAC access type 0100 B ). This 
operation makes both prefetch channels available. 

FF045240 00000004 Single- word write operation to the Prefetch-Control regis- 

ter of the PS { BXU employing the "Register Request Using 
a Physical Address" IAC (IAC access type 0100 B ). This 
operation makes both prefetch channels available. 

FF041054 00000007 Single-word write operation to the FT1 register of the PS 

BXU employing the "Register Request Using a Physical 
Address" IAC (IAC access type 0100 B ). This operation 
selects the Wait-for-BERL timing, enables the error report- 
ing network, and ensures proper fault-tolerant operation. 

FF045054 00000007 Single- word write operation to the FT1 register of the PSj 

BXU employing the "Register Request Using a Physical 
Address" IAC (IAC access type 0100 B ). This operation 
selects the Wait-for-BERL timing, enables the error report- 
ing network, and ensures proper fault-tolerant operation. 



3B. FF0410E4 



00000500 Single-word write operation to the Module-Error-ID regis- 
ter of the PS BXU employing the "Register Request Using 
a Physical Address" IAC (IAC access type 0100 B ). This op- 
eration sets the Confinement-ID field to a value of five and 
the Source-ID field to a value of zero. 



FF0450E4 00000503 Single- word write operation to the Module-Error-ID regis- 

ter of the PS 1 BXU employing the "Register Request Using 
a Physical Address" IAC (IAC access type 0100 B ). This 
operation sets the Confinement-ID field to a value of five and 
the Source-ID field to a value of one. 
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Step Address 
(Hex) 



Data 
(Hex) 



Action 



3C. FF0410E8 0000000A Single- word write operation to the Bus-Error-ID register of 

the PS BXU employing the "Register Request Using a 
Physical Address" IAC (IAC access type 0100 B ). This 
operation sets the Confinement-ID field to a value of zero 
and the Source-ID field to a value of five. 

FF0450E8 00000 1 0B Single- word write operation to the Bus-Error-ID register of 

the PS j BXU employing the "Register Request Using a 
Physical Address" IAC (IAC access type 0100 B ). This 
operation sets the Confinement-ID field to a value of one and 
the Source-ID field to a value of five. 



4A. FF0410C4 
FF0450C4 

4B. 

5. 

6. FF0000E0 



00000400 Single- word write operation to the Spouse-ID register of the 
PS BXU employing the "Register Request Using a Physical 
Address" IAC (IAC access type 0100 B ). This operation sets 
the Spouse-ID field to a value of four. 

00000400 Single-word write operation to the Spouse-ID register of the 
PS j BXU employing the "Register Request Using a Physical 
Address" IAC (IAC access type 0100 B ). This operation sets 
the Spouse-ID field to a value of four, inge 

This step was done in the parameterization phase. 

This step was done in the identification phase. 

0000000E Single-word write operation to the QMR register of the PP 
and PPj BXUs using the "Register Request From the L-bus" 
IAC (IAC access type 0000 B ). This operation sets the 
Married, and Toggle-Primary I Shadow control bits. Note 
that the write enable for the Married bit is turned on. 



FF04 1 0E0 0000003E Single- word write operation to the QMR register of the PS 

BXU employing the "Register Request Using a Physical 
Address" IAC (IAC access type 0100 B ). This operation sets 
the Married, Toggle-Primary I Shadow, and Shadow control 
bits . Note that the write enable for the Shadow and Married 
bits is turned on. 
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Step Address Data 

(Hex) (Hex) 



Action 



FF0450E0 0000003E Single-word write operation to the QMR register of the PS, 

BXU employing the "Register Request Using a Physical 
Address" IAC (IAC access type 0100 B ). This operation sets 
the Married, Toggle-Primary/Shadow, and Shadow control 
bits. Note that the write enable for the Shadow and Married 
bits is turned on. 

7. No action required because the AP-Mask and AP-Match 

registers are not used in this sample configuration. 

Register Summary of the BXUs in the Sample Configuration 

Tables 14-3 and 14-4 summarize the final identification values for the various registers. Because 
some of the registers of the BXU are written in several steps, the final value of the register shown 
in the register summaries of each BXU may not agree with the values shown in the individual steps 
previously described. Note that any value shown for a master BXU is identical for its checker BXU. 
This matching o ccurs aut omatically because of two actions: the checker operates in lockstep with the 
master, and the INITID pins of the master and checker pairs are connected together. 
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Table 14-3: Summary of Identification Values of Each BXU on AP-Bus 



Register 


Field 


BXU' 


*Y 


PS 


l/OPo 


i/os 


Logical-ID 


Unit-ID 


00 0000 B 
(0) 


00 0000b 
(0) 


00 0001b 

(1) 


00 0001 B 

(1) 


Physical-ID 


Component-ID 


0000 B 
(0) 


0001 B 

(1) 


001 B 
(2) 


0011 B 
(3) 


Arbitration-ID 


Drive 


oo B 

(0) 


oo B 

(0) 


oo B 

(0) 


00 B 
(0) 


Count 


0000 B 
(0) 


0000b 
(0) 


0001 B 

(1) 


0001 B 

(1) 


Module-Error-ID 


Confinement-ID 


0000 0100 B 
(4) 


0000 0101 B 
(5) 


0000 0110 B 
(6) 


0000 0111 B 
(7) 


Source-ID 


000 0000b 
(0) 


000 0000b 
(0) 


000 0000b 
(0) 


000 0000b 
(0) 


Bus-Error-ID 


Confinement-ID 


0000 0000 B 
(0) 


0000 0000b 
(0) 


0000 0000b 
(0) 


0000 0000b 
(0) 


Source-ID 


000 0100b 
(4) 


000 0101 B 
(5) 


000 0110b 
(6) 


000 0111b 
(7) 


Spouse-ID 


Spouse-ID 


0000 0101b 
(5) 


0000 0100b 
(4) 


0000 0111 B 
(7) 


0000 0110b 
(6) 



NOTE: 

* The numerical value within the parentheses is the decimal equivalent of the binary number. 
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Table 14-4: Summary of Identification Values of Each BXU on AP-Bus, 



Register 


Field 


BXU' 


PPi 


PS, 


l/OP, 


l/OS, 


Logical-ID 


Unit-ID 


00 0000 B 
(0) 


00 0000b 
(0) 


00 0001 B 

(1) 


00 0001b 

(1) 


Physical-ID 


Component-ID 


0000 B 
(0) 


0001b 

(1) 


0010b 
(2) 


0011b 
(3) 


Arbitration-ID 


Drive 


oo B 

(0) 


00 B 
(0) 


00 B 
(0) 


00 B 
(0) 


Count 


0000b 
(0) 


0000b 
(0) 


0001 B 

(1) 


0001 b 

(1) 


Module-Error-ID 


Confinement-ID 


0000 0100b 
(4) 


0000 0101 B 
(5) 


0000 0110b 
(6) 


0000 0111 B 
(7) 


Source-ID 


000 0001 B 

(1) 


000 0001 B 

(1) 


000 0001 B 

(1) 


000 0001 B 

(1) 


Bus-Error-ID 


Confinement-ID 


0000 0001b 

(1) 


0000 0001b 

(1) 


0000 0001 B 

(1) 


0000 0001 B 

(1) 


Source-ID 


000 0100b 
(4) 


000 0101 B 
(5) 


000 0110b 
(6) 


000 0111 B 
(7) 


Spouse-ID 


Spouse-ID 


0000 0101b 
(5) 


0000 0100b 
(4) 


0000 0111 B 
(7) 


0000 0110b 
(6) 



NOTE: 

* The numerical value within the parentheses is the decimal equivalent of the binary number. 
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Register Summary of PP 

The following information summarizes the value of each register in the PP BXU. 



Register 

Arbitration-ID 
AP-Control 
AP-Mask 
AP-Match 
Logical-ID 
Physical-ID 
System-Bus-ID 
LBI-Control 
Mask 
Match 
Mask j 
Match j 
Mask 2 
Match 2 
Private- 
Memory-Mask 
Private- 
Memory-Match 
Cache- 
Configuration 
Processor- 
Priority 
Prefetch- 
Control 

Lock 
FT1 

FRC 
QMR 
FT2 
Maxtime 

Module- 
Error-ID 

Bus-Error-ID 

Error-Log 

Error-Record 

FRC-Splitting- 

Control 

Spouse-ID 



Value Comment 
(Hex) 

00000000 No modification from default values. 

00000000 No modification from default values. 

00000000 No modification from default values. 

00000000 No modification from default values. 

00000000 No modification from default values. 

00000000 No modification from default values. 

00000000 No modification from default values. 

00000030 The INIT-RAM memory recognizer is disabled. 
FFFC0000 

00040001 Enabled with bus recovery, 

00000000 No modification from default values. 

00000000 No modification from default values. 

00000000 No modification from default values. 

00000000 No modification from default values. 

00000000 No modification from default values. 

00000000 No modification from default values. 

00000000 No modification from default values . 

00000000 User must set thePriority field. 

00000004 Prefetch functions enabled. User must send a Start command to 

activate channel. 

00000000 This register is not used in Processor mode. 

00000007 Sets theWait-for-BERL, the Error-Reporting-Enable, and the Iden- 

tify-Device-Disable bits. 

0000000M M is set to a value of one for a master, to a value of zero for Checker. 

0000000A Married bit and the Toggle-Primary I Shadow bit are set. 

00000000 No modification from default values. 

00000000 No modification from default values; minimum transient wait 

selected. 

0000040 1 The Confinement-ID field is set to a value of four, and the Source- 
ID field is set to a value of zero. 

00000009 The Confinement-ID field is set to a value of zero, and the Source- 
ID field is set to value of four. 

00000000 No modification from default values. 

00?????? No modification from default values. 

00000000 No modification from default values. 

00000500 The Spouse-ID field is set to a value of five. 
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Register Summary of PS 

The following information summarizes the value of each register in the PS BXU. 

Comment 



No modification from default values. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

The Component-ID field is set to a value of one. 

No modification from default values. 

The INIT-RAM memory recognizer is disabled. 

Enabled with bus recovery. 
No modification from default values. 
No modification from default values. 
No modification from default values. 
No modification from default values. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

User must set the Priority field. 

Prefetch functions enabled. User must send a Start command to 

activate channel. 

This register is not used in Processor mode. 

Sets the Wait-for-BERL, the Error-Reporting-Enable, and the 

Identify-Device-Disable bits. 

M is set to a value of one for a master, to a value of zero for Checker. 

Married bit, Shadow bit, and the Toggle-Primary I Shadow bit are 

set. 

No modification from default values. 

No modification from default values; minimum transient wait 

selected. 

The Confinement-ID field is set to a value of five, and the Source- 
ID field is set to a value of zero. 

The Confinement-ID field is set to a value of zero, and the Source- 
ID field is set to a value of five. 



Register 


Value 




(Hex) 


Arbitration-ID 


00000000 


AP-Control 


00000000 


AP-Mask 


00000000 


AP-Match 


00000000 


Logical-ID 


00000000 


Physical-ID 


00040000 


System-Bus-ID 


00000000 


LBI-Control 


00000030 


Mask 


FFFC0000 


Match 


00040001 


Maskj 


00000000 


Matchj 


00000000 


Mask 2 


00000000 


Match 2 


00000000 


Private- 




Memory-Mask 


00000000 


Private- 




Memory-Match 


00000000 


Cache- 




Configuration 


00000000 


Processor- 




Priority 


00000000 


Prefetch- 




Control 


00000004 


Lock 


00000000 


FTI 


00000007 


FRC 


0000000M 


QMR 


0000001A 


FT2 


00000000 


Maxtime 


00000000 


Module- 




Error-ID 


00000500 


Bus-Error-ID 


0000000A 
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Register Value Comment 
(Hex) 

Error-Log 00000000 No modification from default values. 
Error-Record 00?????? No modification from default values. 
FRC -Splitting- 
Control 00000000 No modification from default values. 
Spouse-ID 00000400 The Spouse-ID field is set to a value of four. 
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Register Summary of l/OP 

The following information summarizes the value of each register in the I/OP BXU. 

Comment 

No modification from default values. 

This register is enabled to recognize addresses 00040000 to 

0007FFFF (256KB window). 

The Unit-ID field is set to a value of one. 

The Component-ID field is set to a value of two. 

No modification from default values. 

The INIT-RAM memory recognizer is disabled and the BXU acts 

as primary bus master on the L-bus. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

This register is not used by the I/O module. 

No modification from default values. 

This register is not used in Processor mode. 

Sets the Wait-for-BERL, thcError-Reporting-Enable, and the Iden- 

tify-Device-Disable bits. 

M is set to a value of one for a master, to a value of zero for Checker. 

Married bit and the Toggle-Primary I Shadow bit are set. 

No modification from default values. 

No modification from default values; minimum transient wait 

selected. 

The Confinement-ID field is set to a value of six, and the Source- 
ID field is set to a value of zero. 

The Confinement-ID field is set to a value of zero, and the Source- 
ID field is set to a value of six. 
No modification from default values. 



Register 


Value 




(Hex) 


Arbitration-ID 


00000001 


AP-Control 


00000000 


AP-Mask 


FFFC0000 


AP-Match 


00040001 


Logical-ID 


00040000 


Physical-ID 


00080000 


System-Bus-ID 


00000000 


LBI-Control 


000000F0 


Mask 


00000000 


Match 


00000000 


Maskj 


00000000 


Match 1 


00000000 


Mask 2 


00000000 


Match 2 


00000000 


Private- 




Memory-Mask 


00000000 


Private- 




Memory-Match 


00000000 


Cache- 




Configuration 


00000000 


Processor- 




Priority 


00000000 


Prefetch- 




Control 


00000000 


Lock 


00000000 


FT1 


00000007 


FRC 


0000000M 


QMR 


0000000A 


FT2 


00000000 


Maxtime 


00000000 


Module- 




Error-ID 


00000600 


Bus-Error-ID 


oooooooc 


Error-Log 


00000000 
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Register 



Value 
(Hex) 



Error-Record 00?????? 

FRC-Splitting- 

Control 00000000 

Spouse-ID 00000700 



Comment 



No modification from default values. 

No modification from default values. 

The Spouse-ID field is set to a value of seven. 
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Register Summary of l/OS 

The following information summarizes the value of each register in the I/OS BXU. 

Comment 



No modification from default values. 

This register is enabled to recognize addresses 00040000 to 

0007FFFF (256KB window). 

The Unit-ID field is set to a value of one. 

The Component-ID field is set to a value of three. 

No modification from default values. 

The INIT-RAM memory recognizer is disabled and the BXU acts 

as primary bus master on the L-bus. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

This register is not used by the I/O module. 

No modification from default values. 

This register is not used in Processor mode. 

Sets the Wait-for-BERL, the Error-Reporting-Enable, and the 

Identify-Device-Disable bits. 

M is set to a value of one for a master, to a value of zero for Checker. 

Married bit, Shadow bit, and the Toggle-Primary I Shadow bit are 

set. 

No modification from default values. 

No modification from default values; minimum transient wait 

selected. 

The Confinement-ID field is set to a value of seven, and the 
Source-ID field is set to a value of zero. 



Register 


Value 




(Hex) 


Arbitration-ID 


00000001 


AP-Control 


00000000 


AP-Mask 


FFFC0000 


AP-Match 


00040001 


Logical-ID 


00040000 


Physical-ID 


ooocoooo 


System-Bus-ID 


00000000 


LBI-Control 


000000F0 


Mask 


00000000 


Match 


00000000 


Mask; 


00000000 


Matchj 


00000000 


Mask 2 


00000000 


Match 2 


00000000 


Private- 




Memory-Mask 


00000000 


Private- 




Memory-Match 


00000000 


Cache- 




Configuration 


00000000 


Processor- 




Priority 


00000000 


Prefetch- 




Control 


00000000 


Lock 


00000000 


FTI 


00000007 


FRC 


0000000M 


QMR 


0000001A 


FT2 


00000000 


Maxtime 


00000000 


Module- 
Error-ID 


00000701 
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Register 



Error-Log 
Error-Record 
FRC -Splitting- 
Control 
Spouse-ID 



Value 
(Hex) 



Bus-Error-ID 0000000F 



00000000 
00?????? 

00000000 
00000600 



Comment 



The Confinement-ID field is set to a value of zero, and the Source- 
ID field is set to a value of seven. 
No modification from default values. 
No modification from default values. 

No modification from default values. 
The Spouse-ID field is set to a value of six. 
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Register Summary of PP 1 

The following information summarizes the value of each register in the PPj BXU. 

Comment 



No modification from default values. 
No modification from default values. 
No modification from default values. 
No modification from default values. 
No modification from default values. 
No modification from default values. 

The INIT-RAM memory recognizer is disabled. 

Enabled with bus recovery. 
No modification from default values. 
No modification from default values. 
No modification from default values. 
No modification from default values. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

User must set the Priority field. 

Prefetch functions enabled. User must send a Start command to 

activate channel. 

This register is not used in Processor mode. 

Sets the Wait-for-BERL, the Error-Reporting-Enable, and the 

Identify -Device-Disable bits. 

M is set to a value of one for a master, set to a value of zero for 

Checker. 

Married bit and the Toggle-Primary/ Shadow are set. 

No modification from default values. 

No modification from default values; minimum transient wait 

selected. 

The Confinement-ID field is set to a value of four, and the Source- 
ID field is set to a value of one. 

The Confinement-ID field is set to a value of one, and the Source- 
ID field is set to value of four. 
No modification from default values. 



Register 


Value 




(Hex) 


Arbitration-ID 


00000000 


AP-Control 


00000000 


AP-Mask 


00000000 


AP-Match 


00000000 


Logical-ID 


00000000 


Physical-ID 


00000000 


System-Bus-ID 


00004000 


LBI-Control 


00000030 


Mask 


FFFC0000 


Match 


00040001 


Maskj 


00000000 


Match] 


00000000 


Mask 2 


00000000 


Match 2 


00000000 


Private- 




Memory-Mask 


00000000 


Private- 




Memory-Match 


00000000 


Cache- 




Configuration 


00000000 


Processor- 




Priority 


00000000 


Prefetch- 




Control 


00000004 


Lock 


00000000 


FT1 


00000007 


FRC 


0000000M 


QMR 


0000000A 


FT2 


00000000 


Maxtime 


00000000 


Module- 




Error-ID 


00000402 


Bus-Error-ID 


00000108 


Error-Log 


00000000 
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Register Value Comment 

(Hex) 

Error-Record 00?????? No modification from default values. 

FRC-Splitting- 

Control 00000000 No modification from default values. 

Spouse-ID 00000500 The Spouse-ID field is set to a value of five. 
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Register Summary of PS t 

The following information summarizes the value of each register in the PSj BXU. 

Comment 



No modification from default values. 
No modification from default values. 
No modification from default values. 
No modification from default values. 
No modification from default values. 
The Component-ID field is set to a value of one. 

The INIT-RAM memory recognizer is disabled. 

Enabled with bus recovery. 
No modification from default values. 
No modification from default values. 
No modification from default values. 
No modification from default values. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

User must set the Priority field. 

Prefetch functions enabled. User must send a Start command to 

activate channel. 

This register is not used in Processor mode. 

Sets the Wait-for-BERL, the Error-Reporting-Enable, and the 

Identify-Device-Disable bits. 

M is set to a value of one for a master, to a value of zero for Checker. 

Married bit, Shadow bit, and the Toggle-Primary I Shadow bit are 

set. 

No modification from default values. 

No modification from default values; minimum transient wait 

selected. 

The Confinement-ID field is set to a value of five, and the Source- 
ID field is set to a value of one. 

The Confinement-ID field is set to a value of one, and the Source- 
ID field is set to value of five. 



Register 


Value 




(Hex) 


Arbitration-ID 


00000000 


AP-Control 


00000000 


AP-Mask 


00000000 


AP-Match 


00000000 


Logical-ID 


00000000 


Physical-ID 


00040000 


System-Bus-ID 


00004000 


LBI-Control 


00000030 


Mask 


FFFC0000 


Match 


00040001 


Mask; 


00000000 


Match] 


00000000 


Mask 2 


00000000 


Match 2 


00000000 


Private- 




Memory-Mask 


00000000 


Private- 




Memory-Match 


00000000 


Cache- 




Configuration 


00000000 


Processor- 




Priority 


00000000 


Prefetch- 




Control 


00000004 


Lock 


00000000 


FTI 


00000007 


FRC 


0000000M 


QMR 


0000001A 


FT2 


00000000 


Maxtime 


00000000 


Module- 
Error-ID 


00000503 


Bus-Error-ID 


0000010B 
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Register 



Error-Log 

Error-Record 

FRC-Splitting- 

Control 

Spouse-ID 



Value 
(Hex) 

00000000 
00?????? 

00000000 
00000400 



Comment 



No modification from default values. 
No modification from default values. 

No modification from default values. 

The Spouse-ID field is set to a value of four. 
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Register Summary of l/OP 1 

The following information summarizes the value of each register in the I/OP r BXU. 

Register Value Comment 

(Hex) 



Arbitration-ID 0000000 1 

AP-Control 00000000 

AP-Mask FFFC0000 

AP -Match 00040001 

Logical-ID 00040000 

Physical-ID 00080000 

System-Bus-ID 00000000 

LBI-Control 000000F0 



Mask 


00000000 


Match 


00000000 


Maskj 


00000000 


Matchj 


00000000 


Mask 2 


00000000 


Match 2 


00000000 


Private- 




Memory-Mask 


00000000 


Private- 




Memory-Match 


00000000 


Cache- 




Configuration 


00000000 


Processor- 




Priority 


00000000 


Prefetch- 




Control 


00000000 


Lock 


00000000 


FT1 


00000007 


FRC 


0000000M 


QMR 


0000000A 


FT2 


00000000 


Maxtime 


00000000 


Module- 




Error-ID 


00000603 



Bus-Error-ID 00000 1 OD 



No modification from default values. 

This register is enabled to recognize addresses 00040000 to 

0007FFFF (256KB window). 

The Unit-ID field is set to a value of one. 

The Component-ID field is set to a value of two. 

No modification from default values. 

The INIT-RAM memory recognizer is disabled and the BXU acts 

as primary bus master on the L-bus. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

This register is not used by the I/O module. 

No modification from default values. 

This register is not used in Processor mode. 

Sets the Wait-for-BERL bit, the Error-Reporting-Enable, and the 

Identify -Device-Disable bits. 

M is set to a value of one for a master, to a value of zero for Checker. 

Married bit and theToggle-Primaryi Shadow bit are set. 

No modification from default values. 

No modification from default values; minimum transient wait 

selected. 

The Confinement-ID field is set to a value of six, and the Source- 
ID field is set to a value of one. 

The Confinement-ID field is set to a value of one, and the Source- 
ID field is set to a value of six. 
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Register 



Error-Log 

Error-Record 

FRC-Splitting- 

Control 

Spouse-ID 



Value 
(Hex) 

oooboooo 

00?????? 

00000000 
00000700 



Comment 



No modification from default values. 
No modification from default values. 

No modification from default values. 

The Spouse-ID field is set to a value of seven. 
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Register Summary of l/OS 1 

The following information summarizes the value of each register in the I/OSj BXU. 

Comment 

No modification from default values. 

This register is enabled to recognize addresses 00040000 to 

0007FFFF (256KB window). 

The Unit-ID field is set to a value of one. 

The Component-ID field is set to a value of three. } 

The INIT-RAM memory recognizer is disabled and the BXU acts 

as primary bus master on the L-bus. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

No modification from default values. 

This register is not used by the I/O module. 

No modification from default values. 

This register is not used in Processor mode. 

Sets the Wait-for-BERL, the Err or -Rep or ting-Enable, and the 

Identify-Device-Disable bits. 

M is set to a value of one for a master, to a value of zero for Checker. 

Married bit, Shadow bit, and the Toggle-Primary I Shadow bit are 

set. 

No modification from default values. 

No modification from default values; minimum transient wait 

selected. 

The Confinement-ID field is set to a value of seven, and the Source- 
ID field is set to a value of one. 

The Confinement-ID field is set to a value of one, and the Source- 
ID field is set to a value of seven. 



Register 


Value 




(Hex) 


Arbitration-ID 


00000001 


AP-Control 


00000000 


AP-Mask 


FFFC0000 


AP-Match 


00040001 


Logical-ID 


00040000 


Physical-ID 


ooocoooo 


System-Bus-ID 


00040000 


LBI-Control 


000000F0 


Mask 


00000000 


Match 


00000000 


Maskj 


00000000 


Match; 


00000000 


Mask 2 


00000000 


Match 2 


00000000 


Private- 




Memory-Mask 


00000000 


Private- 




Memory-Match 


00000000 


Cache- 




Configuration 


00000000 


Processor- 




Priority 


00000000 


Prefetch- 




Control 


00000000 


Lock 


00000000 


FT1 


00000007 


FRC 


0000000M 


QMR 


0000001A 


FT2 


00000000 


Maxtime 


00000000 


Module- 




Error-ID 


00000702 


Bus-Error-ID 


0000010E 
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Register 



Error-Log 

Error-Record 

FRC-Splitting- 

Control 

Spouse-ID 



Value 
(Hex) 

00000000 
00?????? 

00000000 
00000600 



Comment 



No modification from default values. 
No modification from default values. 

No modification from default values. 
The Spouse-ID field is set to a value of six. 
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SUMMARY 

Initialization of the 80960 system consists of three phases: default, identification, and parameteri- 
zation. After RESET is applied, the registers of the BXU are set to a predetermined value listed in 
Appendix A for each register. These default values allow a small system to operate without 
additional initialization. These values can be changed during the identification and parameterization 
phases. The identification phase is used to assign a Component-ID field in the Physical-ID register. 
During the parameterization phase, other registers, such as the Logical-ID register can be modified 
by using the I AC register requests. 

The initialization of QMR modules requires assistance of another module or agent. This chapter 
showed an example of initializing and marrying two FRC pairs using another agent. 
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CHAPTER 15 
FAULT-TOLERANT I/O CONSIDERATIONS 

The CPU and memory modules operate synchronously with the system clock. If the I/O module 
operates synchronously to the system clock, the same detection mechanisms and recovery techniques 
that are employed by the CPU and memory modules can also be used by the I/O module. Many 
I/O systems, however, such as an Ethernet™ controller, or an interrupt controller, function asynchro- 
nously to the system clock. This chapter presents some of the considerations for asynchronous fault- 
tolerant I/O systems, which include the following items: 

• Interface to an asynchronous I/O system 

• Interrupt handling 

OVERVIEW 

An I/O system can be duplicated to form a master/checker pair if it can operate synchronously to the 
system clock and can produce identical responses when it is replicated. If both of these conditions 
are satisfied, the master/checker pair can be married to another master/checker pair. This marriage 
forms a QMR I/O module, as shown in Figure 15-1. Thus, synchronous I/O modules can take full 
advantage of the fault-tolerant facilities of the BXU. The error detection mechanisms, error 
reporting, and recovery mechanisms described in the previous chapters can be used. 
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Figure 15-1: Synchronous I/O Interface 

Most I/O systems, however, are asynchronous or cannot reproduce identical output signals when 
replicated. For example, consider an I/O system that contains an analog shaft position. To form an 
FRC pair, two transducers are used, one for each BXU. For identical shaft positions, the output of 
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each transducer may vary just enough to cause an error if the outputs were compared. Other examples 
include an Ethernet™ controller, which cannot guarantee identical output signals, or an interrupt 
controller where the signals are asynchronous to the system clock. 

An I/O system that operates asynchronously or that cannot reproduce identical signals when it is 
replicated must use a different method to attain fault-tolerant operation. To achieve a fault-tolerant 
I/O module under these conditions, one design approach may interface an FRC pair to a single I/O 
system. The FRC pair forms the boundary for the I/O module confinement area to ensure that the 
data entering the I/O module is error-free. The single I/O system eliminates the problem of similar, 
but not exact output signals from duplicated peripherals. Thus, the design consists of interfacing two 
synchronous L-buses to a single asynchronous I/O system. The next section considers the guidelines 
for designing this type of interface. 

FAULT TOLERANT I/O SYSTEM 

Figure 15-2 shows an example of an asynchronous I/O interface to an 80960 fault-tolerant system. 
This system consists of an I/O peripheral, a synchronizer, two passive I/O interface circuits, two 
comparators, a memory array, two IAC generators, and an FRC master/checker pair. Except for the 
IAC generator logic and memory array, each of the logic blocks above are described in separate 
sections. See Chapter 9 for details on the IAC generator. 

In a typical sequence of events, the I/O device sends data to the local memory and triggers the IAC 
generator to send an "Interrupt" IAC to the 80960MC processor. The 80960MC processor responds 
by performing a read operation on the local I/O memory. 

FRC Pair 

By pairing two BXUs, the FRC detection mechanism c an be u s ed to c o mpare AP-bu s si gnals for data 
inconsistencies. The FRC pair checks the AD 31 -AD , SPEC 5 -SPEC , MODCHK, and BOUT lines 
as the signals flow to and from the AP-bus ensuring that the signals are correct. The address and data 
from the I/O device, however, may not be able to be checked if these signals originate from a single 
I/O device, rather than two identical I/O devices (the data cannot be compared). Consequently, an 
I/O system that is not replicated should not be able to access the AP-bus. 

Passive Interface Circuit 

To prevent undetected corrupt data flowing from the BXU onto the AP-bus, the passive interface 
circuit does not permit the I/O peripheral to access the L-bus and hence, the AP-bus. This statement 
does not imply that the I/O module must be polled by the 80960MC processor. It simply means that 
any data transmitted to the AP-bus must originate in the synchronous confinement areas of the master 
and checker, such as an IAC Generator. Two L-buses are used in order to ensure that logic failures 
on the L-bus are detected. Thus, there is a passive interface circuit for each L-bus. 
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Figure 15-2: Asynchronous I/O Interface 



Synchronizer 



The synchronizer makes the signal assertions from the I/O device synchronous to the system clock. 
All of the synchronizer outputs are connected to both the master and checker. Otherwise, FRC 
failures may occur at the AP-bus interface if the signals arrive at the interface at slightly different 
times. Synchronizers can be replicated provided that the signal outputs are connected to both the 
master and checker. 



Figure 15-3 shows an example of a synchronizer for the READY signa l. The tw o stages of the 
synchronization circuit reduces the metastable effects. Both synchronous READY signals must be 
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connected to both BXUs. Two READY signals provide fault detection, more READY signals would 
provide fault mask ing. The FRC interface should provide a time-out capability to guard against 
receipt of only one READY signal. 
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Figure 15-3: Synchronization Circuit 

The outputs of each synchronizer should be connected to both the master and checker because the 
logic of each synchronizer can respond to the same clock signal at different times. For example, 
consider the synchronizer in Figure 15-4. Each synchronizer requires a setup time of 10 ns. If a signal 
arrives in less than 10 ns, one logic circuit may respond, while the other may not, due to variations 
in the device response times. If both outputs were not sent to both the master and checker pair, the 
master may have received a synchronizing command, while the checker may not have. Conse- 
quently, an error report would be generated. By routing both signals to each BXU in the master/ 
checker pair, an error is not generated because each component receives the same sequence of 
signals. 

I/O System 

A fault-tolerant I/O system must ensure two actions: it must not corrupt the rest of the system by 
sending inaccurate data, and it must perform the intended function (for example, a motor must move 
the commanded degrees). 

To comply with the first requirement, FRC pairs can be used to detect any failure on the L-bus or by 
the BXU in the I/O module. The passive interface and software checks can be used to prevent 
addresses and data (passively generated by the I/O module) from corrupting the system. The I/O 
module can be active (initiate requests without being commanded by a processor or other device), 
if it issues a request from a redundant source. For example, two IAC generators can be located on 
each L-bus (one generator per bus). In this manner, the I/O system cannot place data at a random 
address in which software cannot check. 
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The second requirement is a system design issue that applies fault-tolerant design techniques to the 
I/O destination subsystems. Because there is a broad range of I/O systems available, this fault- 
tolerant design consideration is beyond the scope of this manual. 
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Figure 15-4: Synchronization Circuit Example 



Comparators 



The comparators perform clock-by-clock checking on every signal passing across the passive 
interface (this also implies that the passive interface must be synchronous). The comparators 
themselves need not be fault-tolerant since they are replicated. However, they should have some 
fault-tolerant way of reporting errors. For example, if a master comparator detects an error caused 
by the master BXU, and then reports the error to the master BXU, the error report may not be 
propagated. Errors at the I/O confinement area interface should be reported to both the master and 
checker to ensure errors get reported in case of an L-bus failure. 

The comparator ensures that output data and the control signals are correct. The comparator checks 
signals on a clock-by-clock basis. Under normal operation, the comparators check the information 
as it flows to or from the L-bus from or to the I/O-bus. If the I/O system and BXUs place data on 
each side of the comparator, potential conflicts may occur. The se confl icts are resolved by designing 
communication lines between each comparator, similar to the BOUT line between the FRC BXUs. 
This connection ensures that when activity on the L-bus and I/O-bus coexist, an error is not generated. 

Design Issues 

There are several issues that arise when designing a fault-tolerant I/O system. These issues are totally 
dependent upon the design goals and implementation. Consequently, the following issues are listed 
to serve as a checklist. 



15-5 



inteT 



FAULT-TOLERANT I/O CONSIDERATIONS 



The fault-tolerant design should be kept in balance. For example, if the I/O system has a long 
mean time between failure (MTBF) compared to the interface circuit, then it makes sense to 
implement a fault-tolerant interface circuit. If, however, the MTBF of the I/O system is low, then 
it may not be worth the effort to ensure that the interface circuit has a high MTBF. 

A policy should be devised concerning latent errors and diagnostic tests. The design may 
incorporate some method to exercise the fault-tolerant circuitry to ensure that it operates 
correctly. 



• Any read or write request to the I/O module must be terminated at the BXU with a valid READY 
signal. If the I/O system fails during a read or write operation, the BXU sends a RP YDEF signal 
indefinitely because it does not receive a READY signal. This action effectively uses one-third 
of the total AP-bus bandwidth. 

• Decisions need to be made whether to have one interface circuit drive the I/O-bus at all times 
or alternate the passive I/O interface circuits on a clock-by-clock basis. 

QMRI/O 

The QMR configuration is based upon marrying two identical synchronous FRC pairs by software. 
Because asynchronous I/O systems cannot be truly configured as FRC pairs, the QMR configuration 
described in Chapters 10 through 13 cannot be used. Alternative methods are needed to construct 
a fault-tolerant I/O system using asynchronous peripherals. 

Figure 15-5 shows one alternative that implements two FRC pairs each of which interfaces to an 1/ 
O system. For this configuration, the two FRC pairs function as independent I/O systems. Software 
is responsible for the comparison of signals, detection of errors, and recovery from the errors. 

To illustrate the use of non-married FRC pairs, Figure 15-6 shows an example of a motor controller 
with redundant transducers and control switches. The analog shaft position signals from the 
transducers are applied to the synchronizers. The outputs of the two synchronizers of each FRC pair 
are connected to both passive interface circuits to ensure that the BXUs remain in lockstep. The 
passive interface should be designed to be fail-safe (i.e., it automatically opens the switch when a 
module failure is detected). The serial/parallel configuration of the switches provides FRC checking 
on the I/O passive outputs (the bus switches must be closed to turn on motor). This setup guards 
against a failure of a single module because there is a redundant pair of switches to turn off the motor. 

The 80960MC processor can either poll the I/O system, or the I/O system can send an "Interrupt" I AC 
to the 80960MC processor. In either case, the 80960MC can read the data at the I/O system to perform 
software voting or averaging. Based upon the results, the 80960MC processor issues a command to 
open or close the motor switches. 

HANDLING INTERRUPTS IN FAULT-TOLERANT SYSTEMS 

Interrupts within a fault-tolerant system can be categorized into two classes: those generated by 
devices local to the interrupted processor, such as timers; and those generated by remote devices, 
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such as sensors. The former are generally synchronized with the system clock, while the latter are 
inherently asynchronous. 
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Figure 15-5: Alternative to a QMR I/O System 

The synchronous interrupt sources can be replicated and designed to use the on-chip interrupt 
controllers in each of the replicated processors. This method of handling synchronous interrupts 
preserves the system's fault tolerance. 

Asynchronous interrupt sources, however, cannot be connected to the interrupt controllers of 
replicated processors without compromising the system's fault tolerance. This is because interrupts 
arriving at slightly different times could cause the replicated processors to go out of synchronization. 
Moreover, the differences in setup and response times of the duplicated interrupt logic could cause 
the processors to go out of synchronization. 
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Figure 15-6: I/O Example of Soft QMR I/O System 
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Consequently, asynchronous interrupts must be isolated from the rest of the fault-tolerant 80960 
system. The isolation can be attained by generating the "Interrupt" I AC on the L-bus. This I AC 
causes the 80960MC processor to respond to the interrupt assuming that it has a higher priority than 
the current process. Thus, an agent on the I/O subsystem L-bus must accept interrupts and generate 
their associated IAC messages. This function can be performed by another 80960MC, or by an IAC 
Generator if the power of the processor is not required (see Chapter 9, "The IAC Generator" section 
for more details on this circuit). 

SUMMARY 

The fault- tolerant facilities of the BXU are used for synchronous, reproducible I/O systems. For 
asynchronous or non-reproducible I/O systems, a fault-tolerant configuration can be built by using 
non-married FRC pairs to form a redundant I/O system with data comparisons done by software. The 
FRC pairs check the data to the AP-bus and software routines must ensure that no corrupt data leaves 
the I/O system. The interface between the I/O system and FRC pair includes a passive circuit and 
a synchronizer. An active I/O system must guarantee that no corrupt data or addresses are transmitted 
to the AP-bus. Two FRC pairs can be used to form a redundant I/O system with software performing 
the error detection and recovery. 
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APPENDIX A 
BXU REGISTERS AND COMMANDS 

The BXU has various programmable registers and commands that provide a flexible interface to the 
systems software. Table A-l shows a summary of all the registers and commands in the BXU, and 
table A-2 provides a memory-map of all the BXU's registers. This appendix provides the details of 
each register and command, which are listed in alphabetical order (the name of the register or 
command is listed on the top of the page). the page). 

If a software program needs to modify a single register bit, it normally reads the entire contents of 
the register, modifies the appropriate bit, and writes back the entire contents. This process functions 
properly as long as hardware does not modify the register bits while software performs this read- 
modify-write operation. For those register bits that hardware can modify at any time, the BXU 
provides a Write-Enable control bit that protects the specified bit from changing during the read- 
modify-write operation. In this way, if hardware changes a bit in the register during a read-modify- 
write operation, the software does not rewrite the protected bit. 

The Write-Enable is turned on or off by the value of the data word in the write operation. 
Consequently, during a read operation, the Write-Enable bit does not provide data. The data returned 
in this bit position is equal to the corresponding address bit that was used to access the register request. 
Thus, bits designated as "don't care" bits may have different values when the register is read using 
different types of I AC requests. To insure consistent operation, software should mask all bit positions 
that are "don't care" before using the data. 

Typically, the BXU registers are written during initialization and system configuration. Registers 
can be modified during normal operation, but care must be exercised to maintain correct operation 
of the BXU. The following lists specify the restrictions on register write operations. See the register 
definitions in this appendix for descriptions of the contents of the registers and for any effects that 
occur because of register modifications. 

The registers listed below may be written at any time without any restrictions, although the optimum 
ime to write to the Arbitration-ID register is when there are no outstanding requests. 

Arbitration-ID (write-only) 

COM 

I AC -Message-Buffer 

Processor-Priority 

Test-Detection 

Test-Type 



A-1 



iny 



APPENDIX A 



Table A-1 : Summary of BXU Registers and Commands 



AP-Bus Interface (2) 


Register 


Address 


PHYSICAL-ID 


040 H 


LOGICAL-ID 


044 h 


COMPONENT-SPECIFIER (1) 


048 h 


ARBITRATION-ID (1) 


04V 


COM 


04C H 


AP-CONTROL 


050 H 


FT1 


054 H 


MAXTIME 


058 H 


FRC-SPLITTING-CONTROL 


05C H 


TEST-DETECTION 


060 H 


FRC 


064 H 


AP-MATCH 


068„ 


AP-MASK 


06C H 



NOTES: 

(1) The component specifier (read only) and the 
Arbitration-ID (write-only) are physically two 
separate registers that share a common address. 

(2) The AP-Bus interface registers are the only 
registers that can be addressed using access 
mode. 



L-Bus Interface 


Register 


Address 


PHYSICAL-ID (LOCAL) 


140 H 


LOGICAL-ID (LOCAL) 


144 h 


LBI-CONTROL 


148 h 


SYSTEM-BUS-ID 


14C h 


CACHE-CONFIGURA TION 


150 h 


CACHE-TEST 


154 h 


LOCAL-BUS-TEST 


158 h 


MATCH 


160 H 


MASK 


164 H 


MATCH, 


168 h 


MASK, 


16C h 


MATCH 2 


170„ 


MASK 2 


174 H 


PRIVA TE MEMORY MA TCH 


178 h 


PRIVATE MEMORY MASK 


17C„ 


Command 


Address 


INVALfDATE-CACHE 


15C H 




Memory 


Register 


Address 


LOCK 


300 H 


Externally Mapped 

Registers (Reserved 

for Memory Controller) 


340 H -37C H 
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Table A-1 : Summary of BXU Registers and Commands (cont.) 



IAC Message Support 


Register 


Address 


PROCESSOR-PRIORITY 


000 H 


PROCESSOR-MESSAGE 


010 h -01C h 


PROCESSOR-PRIORITY, 


020 H 


PROCESSOR-MESSAGE, 


030 H -03C H 




Prefetch Buffer Registers 


Channel 


Buffer 


Word 


Address 











3C0 H 








1 


3C4 H 








2 


3C8 H 








3 


3CC H 





1 





3D0 H 





1 


1 


3D4 H 





1 


2 


3D8 H 





1 


3 


3DC H 










3EO H 







1 


3E4 H 







2 


3E8 H 







3 


3EC H 




1 





3F0 H 




1 


1 


3F4 H 




1 


2 


3F8 H 




1 


3 


3FC H 



Prefetch 


Register 


Address 


PREFETCH-CONTROL 


240 H 



Fault Tolerance 


Register 


Address 


TEST-TYPE 


0C0 H 


SPOUSE-ID 


0C4 H 


QMR 


oeo h 


MODULE-ERROR-ID 


0E4 H 


BUS-ERROR-ID 


0E8 H 


ERROR-LOG 


0EC H 


ERROR-RECORD 


ofo h 


FT2 


0F4 h 


Command 


Address 


TEST-REPORT 


104 H 


PRIMARY-CATASTROPHE 


108 H 


SHADOW-CATASTROPHE 


10C h 


TERMINA TE-PERMANENT 
ERROR-WINDOW 


110 h 


ATTACH-BUS 


114 h 


DETACH-BUS 


118 h 


SYNC-REFRESH 


11C h 
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Table A-2: M80960MC/M82965 Register Map 







Register Contents 




Register Function 


Internal 
Dest Addr (H) 
















(10 Bit Field) 


3 2 


2 1 


1 O 


O 






1 4 


3 6 


5 8 


7 O 


Processor Priority 


000 




PPPP 




PWVF 


* 


004 










* 


008 










* 


OOC 










Processor Message 


010 










Processor Message 


014 










Processor Message 


018 










Processor Message 


01 C 










Processor Priority 1 


020 




PPPP 




PWVF 


* 


024 










* 


028 










* 


02C 










Processor Message 1 


030 










Processor Message 1 


034 










Processor Message 1 


038 










Processor Message 1 


03C 










Physical-ID 


040 




KCCCCC 






Logical-ID 


044 




UUUUUU 






Read = Component-Specifier 


048 Read 






TTT 


TTTSSSSS 


Write = Arbitration ID 


048 Write 








DDCCCC 


Com 


04C 


cccccccc 


CCCCCCCC 


CCCCCCCC 


CCCCCCCC 


AP-Control 


050 








WADRRS 


FT1 


054 








MAC I EB 


Maxtime 


058 








WTMMMM 


FRC-Splitting Control 


05C 








WMWPSF 


Test Detection 


060 








WTFFRR 


FRC 


064 








WRMCTM 


AP-Match 


068 


BBBBBBBB 


BBBBBB 




I I E 


AP-Mask 


06C 


MMMMMMMM 


M M M M M M 




NN 


* 


070 










* 


0BC 










Test Type 


OCO 




TTTT 


TTTTTTTT 


TTTTTTTT 


Spouse ID 


0C4 






SSSSSSSS 






0C8 










* 


0DC 










QMR 


0E0 








PSWSTWME 


Module Error ID 


0E4 






CCCCCCCC 


SSSSSSSP 


Bus Error ID 


0E8 






00 OOOOBB 


SSSSSSSP 


Error Log 


0EC 


EC 


CVTTTTTP 


CCCCCCCC 


SSSSSSSP 


Error Record 


0F0 




VTTTTTP 


CCCCCCCC 


SSSSSSSP 


FT2 


0F4 








S WFCWBR 


* 


0F8 










Test-Report Command 


100 
104 












Primary-Catastrophe Command 


108 






Shadow-Catastrophe Command 


10C 






Term-Perm Err Wind Command 


110 


Data Word Ignored 




Attach-Bus Command 


114 






Detach-Bus Command 


118 






Synch-Refresh Command 


11C 
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Table A-2: M80960MC/M82965 Register Map (cont.) 









Register Contents 




Register Function 


Internal 
Dest Addr (H) 


















(10 Bit Field) 


3 2 


2 1 


1 


O 






1 4 


3 6 


5 8 


7 O 


* 


120 
13C 










Physical-ID (Local) 


140 




KCCCCC 






Logical-ID (Local) 


144 




uuuuuu 






LBI-Control 


148 








E0DBI I NN 


System-Bus-ID (Read-only) 


14C 






ss 




Cache-Configuration 


150 








ETTY I L 


Cache-Test 


154 








WHMHM 


Local-Bus-Test 


158 








MP I ARNTC 


Invalidate-Cache Command 


15C 


C 


ata Word Ignore 


i 




Match 


160 


BBBBBBBB 


BBBBBB 




CSFF 


MaskO 


164 


MMMMMMMM 


M M M M M M 






Match 1 


168 


BBBBBBBB 


BBBBBB 




CSFF 


Maskl 


16C 


MMMMMMMM 


M M M M M M 






Match 2 


170 


BBBBBBBB 


BBBBBB 




CSFF 


Mask 2 


174 


MMMMMMMM 


M M M M M M 






Private Memory Match 


178 


BBBBBBBB 


BBBBBB 




E 


Private Memory Mask 


17C 


MMMMMMMM 


M M M M M M 








180 










* 


23C 










Prefetch-Control 


240 








EAA 


* 


244 










* 


2FC 










Lock 


300 








FTWLLLL 


* 


304 










* 


33C 












3< 


10 


Externally Mapped 














Registers Reserved 














Memory Controller 


37 


C 










* 


380 


* 


3BC 
3C0 










Channel 



Buffer 



Word 







3C4 








1 






3C8 








2 






3CC 








3 






3D0 





1 









3D4 





1 


1 






3D8 





1 


2 




Prefetch Buffer 


3DC 





1 


3 




Registers 


3E0 
3E4 









1 






3E8 







2 






3EC 







3 






3F0 




1 









3F4 




1 


1 






3F8 




1 


2 






3FC 




1 


3 





Indicates address not used 
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The following registers cannot be written during normal operation. Those registers that are not read- 
only can only be written when there is an I AC request pending to this BXU. Violating this restriction 
may cause incorrect or unexpected BXU operations to occur. 

AP-Match 

AP-Mask 

Bus-Error-ID 

Cache -Configuration 

Component-Specifier (read-only) 

Err or -Log 

Error-Record 

FRC-Split-Control 

FT1 

LBI-Control 

Local-Match 

Local-Mask 

Logical-ID 

Module-Error-ID 

Physical-ID 

Prefetch-Control 

System-Bus-ID (read-only) 

The final set of registers consists of special cases. Most of these registers contain bits that may be 
modified and other bits that may not be changed during normal operation. The bits that cannot be 
written during normal operation are either protected with Write-Enable control bits or not writable 
by software (except for the Bus-Switch-Disable bit in the AP -Control register which is not protected 
by the Write-Enable control bit). The parentheses below enclose the control bits or fields that can 
be changed during normal operation. 

AP -Control (only the COM-Altered bit) 

FRC (only the Toggle-M/C and Access-Master/Checker bits) 

FT2 (only the Force-Correct bit) 

Lock (only the Lock-Timed-Out bit) 

Maxtime (only the Maxtime bits for on-line repair) 

QMR (only the Toggle-P/S and Access-P/S bits) 

Spouse-ID (only Spouse-ID field when not married) 
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The access mode of a Logical Address I AC can only be used to address the AP-bus registers. 

The address notation utilized throughout this appendix is the hexidecimal representation of the 
Internal Destination field (bits 0-9) of a Register Request IAC. 
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Descriptions of the Registers and Commands 

AP-Control Register (Address 050 H ) 

This register is the general control and status register of the BXU's AP-bus interface. This register 
may be written during normal operation of the BXU. Figure A- 1 describes the contents of the register 
bits. 
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Figure A-1 : AP-Control Register Contents 



Bit Descriptions 

Bus-Switch-Disable 



Drive-Corn 



1 = Setting this bit to a value of one disables the bus switching 
function. The RPYDEF signal resets the bus time-out and 
places the request currently at the front of the bus pipeline at the 
end of the pipeline. 

= Setting this b it to a valu e of zero enables the bus switching 

function. The RPYDEF signal is used only to disable the bus 
time-out. No reordering of the bus pipeline occurs. 

1 = Software may set this bit to force the COM pin to be asserted. 



= The COM pin is not asserted (if the Faulty bit in the FT2 register 
is set, the BXU drives the COM pin low regardless of this 
setting). 

This bit defaults to a value of zero. 
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COM-Register- Altered 



1 = This setting indicates that the contents of the COM register has 
changed from the time the COM-Register-Altered bit was last 
cleared. 



= This setting indicates that the COM register has not changed. 
This bit defaults to a value of zero. 



Write Enable for 
COM-Register-Altered Bit 



1 = This setting allows a write operation to the COM-Register- 
Altered bit. 



= This setting does not allow a write operation to alter the COM- 
Register-Altered bit. 
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AP-Mask and AP-Match Registers 
(Address 06C H and 068 H ) 

The bits in the AP-Match register are compared against the corresponding bits in the AP-bus address 
cycle, and determine which partition of the address space is recognized by an individual BXU. The 
AP-Match register is also used with the AP-Mask register to control the interleave factor. The AP- 
Mask and AP-Mask registers create an AP-bus address recognizer. 

If a bit in the AP-Mask register is cleared, it causes the corresponding bit position in the AP -Match 
register to be ignored during comparisons. 

The AP-Mask register should always be setup before the AP-Match register is 
enabled. The AP-Mask register defaults to a register value of zero. If the AP-Mask register is not 
changed before the AP-Match register is enabled, then the BXU matches on all AP-bus requests 
(memory and I AC requests). 

Figure A-2 describes the contents of the register fields and bits. 
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Figure A-2: AP-Mask and AP-Match Register Contents 



Field and Control Bit Descriptions 



Recognize 



For the AP-Mask register, each bit(M) in this field that is set causes 
the AP-bus address bit to be compared against the corresponding 
AP-Match register bit. If a bit is cleared, then that bit position is a 
"don't care" during address recognition. 



For the AP-Match register, each bit(B) that is set in this field is 
compared against the corresponding bits in AP-bus address cycles. 
Thus, these bits provide an address for the partition of memory that 
is recognized by this address recognizer. 



Interleave-Control 



These two bits(N) determine the interleaving factor and matching 
for recognizer in the AP-bus Interface. Table A-3 shows the impact 
of the different configurations of these bits. 
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Table A -3: 


Interleave-Control Bit Settings for Different Configurations 


Mask Bits 


Match Bits* 


AD 5 , AD 4 Required for Match* 


00 


XX 


xx (no interleaving) 


01 


xO 


xO (2 way interleaving) 


01 


x1 


x1 (2 way interleaving) 


10t 


XX 


XX 


11 


00 


00 (4 way interleaving) 


11 


01 


01 (4 way interleaving) 


11 


10 


1 (4 way interleaving) 


11 


11 


1 1 (4 way interleaving) 



NOTES: 

* "x" designates a "don't care" value, 
t illegal 



Enable Bit 

(Match Register Only) 



1 = Both the AP-Mask and AP -Match registers are enabled if the 
AP -Inactive bit in the FT1 register and the Faulty bit in the FT2 
register are both zero. If either of these bits is a one, then the AP- 
Mask and AP -Match registers are disabled independent of the 
state of this enable bit. 



= This address recognizer is disabled. 

This bit can also be set by the fault tolerance logic as a result of 
receiving a Sync-Refresh error report. 
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Arbitration-ID Register (Address 048 H ) 

When the BXU needs to issue a request on the AP-bus, it must actively arbitrate for the bus. The way 
in which it arbitrates is determined by the contents of this register. 

The 80960MC processor can only write to this register (a read operation to this address reads the 
contents of the Component-Specifier register). Since only write operations are performed to this 
register, configuration software should establish a relationship between the contents of the Arbitra- 
tion-ID register and some other readable register (for example, the Logical -ID register). Figure 
A-3 describes the contents of the register fields. 
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Figure A-3: Arbitration-ID Register Contents 



Field Descriptions 

Count 



The Count field determines how many time-step intervals ARB 3 is 
asserted during arbitration. The Count field also determines the 
time-step interval in which the BXU asserts the arbitration line 
specified by the Drive field (see Drive field description below). The 
default value for the Count field is fixed in each component. The 
Count field is loaded by an Identify Device Order with bits through 
3 of the second data word. The Count field defaults to a value of zero 
at RESET. 



Drive 



The Drive field determines which arbitration line (ARB 2 , ARB 1? or 
ARB ) to assert during the time-step interval specified by the Count 
field. The Drive field indicates the agent's priority in that time-step 
interval: the ARB line has the highest priority, followed by ARB } , 
then ARB 2 . The mapping of the arbitration lines is shown in Table 
A-4. The Drive field defaults to a value of zero at RESET. 



The default value for the Drive field is set for each component. This 
field is loaded by an Identify Device command with bits 4 and 5 of 
the second data word. 
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Table A-4: Mapping of ARB 2 -ARB to the Drive Bits 



Drive Field 


Arbitration Line 








ARB 





1 


ms, 


1 





ARB 2 


1 


1 


ARB 2 
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Attach-Bus Command (Address 114 H ) 

This command causes the identified bus to be attached to the system and become active. A write 
operation to this register location causes the BXU to generate an Attach-Bus error report, using its 
Bus-Error-ID register for the Location-ID field. The data word sent with the write request is ignored. 
Before sending this command, software should set the Sequence bits in the Match registers of the 
BXU and disable the prefetch function. 

WARNING: The Testing-Enable bit in the Test-Detection register must be set before executing this 
command. Before setting the Testing-Enable bit, there should be a single bit with the value of one 
in the COM register. 
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Bus-Error-ID Register (Address 0E8 H ) 

This register provides the Location-ID field for error reports that originate in an AP-bus confinement 
area. The Location-ID field is comprised of the Source-ID and the Confinement-ID fields of this 
register. Figure A-4 describes the data fields and the control bits of this register. The Confinement- 
ID field is used for the error report message when the error occurs on the bus of this BXU. The 
Confinement-ID field, however, cannot be read or written by means of this register. The Confine- 
ment-ID may be changed by modifying the System-Bus-ID register. 
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Figure A-4: Bus-Error-ID Register Contents 



Field and Control Bit Descriptions 



ID-Parity Bit 



This bit provides even parity for the Source-ID and Confine- 
ment-ID fields in this register. When software loads new values 
into these fields, it must also load thelD-Parity bit to ensure that 
there are an even number of bits with the value of one in this 16- 
bit register. 



Source-ID 



This 7-bit field uniquely identifies a BXU within a confinement 
area. Software must assign a value to this field. Software uses 
the Source-ID field in the Error-Log register to specifically 
identify which BXU (or Master/Checker pair) had issued the 
error report message. 



Confinement-ID 



This 8-bit field is not directly a part of the Bus-Error-ID register 
because the field cannot be read or written using this register 
address. The Confinement-ID field uniquely identifies a con- 
finement area within the system. The BXUs use this value in the 
error report messages to determine which recovery actions to 
take. This register is used when the BXU reports an error in its 
bus confinement area. 
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Bits 8 and 9 of this register are loaded from the AP -Bus-ID field 
in the System-Bus-ID register at RESET. Thus, this field has 
the correct identification value at all times. The ID-Parity bit, 
however, may not be correct at RESET time. This means that 
the BXUs should not send any error reports until the parity bit 
is set correctly by software (the BXUs are able to respond 
correctly to error report messages). This is important software 
consideration for the initialization processor that boots the rest 
of the system.) 
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Cache-Configuration Register (Address 150 H ) 

The control bits in this register together with the control bits of the LBl-Control register determine 
the cache configuration. The Cache -Configuration register specifies the directory organization and 
timing options; the LBl-Control register specifies the interleaving factor. Figure A-5 describes the 
contents of the register fields and control bits. 
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Figure A-5: Cache-Configuration Register Contents 
Field and Control Bit Descriptions 

Line -Configuration Bit 

= This setting specifies four lines per address block. 

1 = This setting specifies eight lines per address block. 

Ignore-Interleave-Control Bit 

= This setting indicates that the interleave control bits from the 

LBl-Control register are used. 

1 = This setting indicates that the LBl-Control register interleaving 

bits are ignored. The cache operates in the non-interleaved 
configuration. 
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WAY -Output Bit 



Timing-Options 



If this bit is set, then the Line-Configuration bit must also be set. This 
restriction is required because the only non-interleaved cache con- 
figuration requires eight lines per address block. 



= When a cache hit occurs, the WYj pin is not asserted while the 

BXU does the cache operation. 

1 = When a cache hit occurs, the WY 2 pin is asserted while the BXU 

does the cache operation. 

If the SRAM is being used for both private memory and cache data 
storage, this bit must be zero. 



These two bits select the timing options for cache read and write 
operations. Bit 4 controls write timing and bit 3 controls read timing. 
In one mode, data is transferred on every clock cycle, while the other 
mode transfers data every other cycle. The slower mode allows the 
cache memory to use slower SRAMs. Table A-5 shows the timing 
selections. 

Table A-5: Timing Selections 



Timing-Option Bits 


Speed Selection 


Bit 4 


Bit 3 


1 


1 


3/6 write timing, 3/6 read timing (fast/fast) 


1 





3/6 write timing, 4/10 read timing (fast/slow) 





1 


4/10 write timing, 3/6 read timing (slow/fast) 








4/10 write timing, 4/10 read timing (slow/slow) 



Enable-Cache Bit 



1 = This setting enables the cache for operation. Other devices on 
the L-bus must now insert at least one wait state in their replies. 
With one wait state the BXU is guaranteed to keep up with the 
worst case coherency traffic on the AP-bus and L-bus. 

= This setting disables the cache for operation. 



A-18 



my 



APPENDIX A 



Table A-6 shows the possible cache configurations based upon the Cache-Configuration and LBI- 
Control registers. 

Table A-6: BXU Cache Configurations 



Configuration 
Number 


Size 


Lines 

Per 

Block 


Ways 


Sets 


BXU 
Interleave 


Cac/)e 

Configuration 

Register 


LBI Control 
Register 


I 


L 


N 


N 


1 


64K 


8 


2 


64 


4 





1 


1 


1 


2 


32K 


8 


2 


64 


2 





1 





1 


3 


16K 


8 


2 


64 


1 





1 


X 





1 


1 


X 


X 


4 


32K 


4 


2 


64 


4 








1 


1 


5 


16K 


4 


2 


64 


2 











1 



Table A-7 shows the address line mapping in the different configurations. The BXU directory logic 
uses the tag, set, and line bits as part of its directory lookup operation. The interleave bits are not 
used. Thus, configurations without interleaving are restricted to operation in single AP-bus systems, 
or dual AP-bus systems where the BXUs on the different buses have the WYj bit set to different 
values. The tag field is 20 bits long (bit 31 through bit 12 ). 

Table A-7: Address Mapping for Different Configurations 



Configuration 


Tag 


Set 


Line 


1 


LAD 31 -LAD 12 


LAD 14 -LAD g 


LAD 8 -LAD 6 


2 


LAD 31 -LAD 12 


LAD 13 -LAD 8 


LAD 7 -LAD 5 


3 


LAD 31 -LAD 12 


LAD 12 -LAD 7 


LAD 6 -LAD 4 


4 


LAD 31 -LAD 12 


LAD 13 -LAD 8 


LAD 7 -LAD 6 


5 


LAD 31 -LAD 12 


LAD 12 -LAD 7 


LAD 6 -LAD 5 
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Table A-8 shows the SRAM address lines. The WY f WY and WD r WD address bits are provided 
by the BXU. All other address bits are from the L-bus signals. 



Table A-8: SRAM Address Lines 



CONFIGURATION 
NUMBER 


ADDRESS LINE 


31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 


8 


7 6 


5 4 3 


2 1 


1 
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111! 
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A 


R 


R 


MR 
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Y 


A 


A 


A 


A 


A 


A 


A 


A 


A 


R 


R 




2,4 + PM 


























Y 


Y 


A 


A 


A 


A 


A 


A 


A 


A 


A 


A 


R 


R 


HI 




3,5 + PM 


























m n 




Y 


Y 


A 


A 


A 


A 


A 


A 


A 


A 


A 


A 


R 


R 


mil 



Notes: 

A — L-Bus Address 
R — Word Bits 
Y — Way Bits 
W$l — No Connection 
PM — Private Memory 
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Cache-Test Register (Address 154 H ) 

This register facilitates the testing of the cache directory and control logic of the BXU. Figure 
A-6 describes the contents of the control bits. 
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Figure A-6: Cache-Test Register Contents 



Bit Descriptions 

Write-Miss 



= This setting indicates that the last cacheable request to this 
BXU was not a write request that missed the cache. 



1 = This setting indicates that the last cacheable request to this 
BXU was a write request that missed the cache. 



Write-Hit 



= This setting indicates that the last cacheable request to this 
BXU was not a write request that hit the cache. 



1 = This setting indicates that the last cacheable request to this 
BXU was a write request that hit the cache. 



Read-Miss 



= This setting indicates that the last cacheable request to this 
BXU was not a read request that missed the cache. 



1 = This setting indicates that the last cacheable request to this 
BXU was a read request that missed the cache. 
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Read-Hit = This setting indicates that the last cacheable request to this 

BXU was not a read request that hit the cache. 

1 = This setting indicates that the last cacheable request to this 
BXU was a read request that hit the cache. 

Way = This setting indicates that the last cacheable request was to 

WAY of the BXU's cache directory. 

1 = This setting indicates that the last cacheable request was to 
WAYj of the BXU's cache directory. 
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COM Register (Address 04C H ) 

This register is used for loading external information, such as the type of board the BXU resides on. 
The register is useful for both initialization and diagnostics. 

The COM register is actually 38 bits wide. Thirty-two bits are visible and 6 bits are hidden. Write 
operations to this register clear the hidden bits. The hidden bits are part of a shift chain used for testing 
the FRC logic in the AP-bus interface. The serial protocol is discussed in Chapter 14. Figure A-7 
describes the contents of the register fields. 
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Figure A-7: COM Register Contents 



Field Descriptions 

System-Data 



This register is used for various testing and parameterization 
functions, as defined in Chapter 14. The default for this register 
is a value of zero. 
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Component-Specifier Register (Address 48 H ) 

The contents of this read-only register are fixed at manufacturing and specify the type and stepping 
of the component. A write operation to this register modifies the Arbitration-ID register. Figure 
A-8 describes the contents of the register fields. 
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Figure A-8: Component-Specifier Register Contents 



Field Descriptions 

Stepping 



This 5 -bit field identifies the stepping number of the compo- 
nent, which is set during manufacturing. 



Component-Type 



This 6-bit field is set during manufacturing and each BXU is 
assigned the same value. This field will be used to distinguish 
a BXU from other future AP-bus components. 
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Detach-Bus Command (Address 118 H ) 

This command causes the identified bus to be detached from the system and become inactive. A write 
operation to this register location causes the BXU to generate Detach-Bus error report on the serial 
error reporting network. The data sent in the data word of the write request is ignored. 

WARNING: The Testing-Enable bit in the Test-Detection register must be set before executing this 
command. Before setting the Testing-Enable bit, there should be a single bit with the value of one 
in the COM register. 
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Error-Log Register(Address OEC H ) 

This register records the type of the most recent error report received and the number of errors that 
have occurred since the last Terminate-Permanent-Error-Window command. This register is the 
primary form of communications between the hardware fault handling logic and the software. 
Because hardware reads and writes to this register, software should not write to it during normal 
system operation. If software does write to this register with the Valid bit not set, then recovery 
mechanism 1 is invoked (see Chapter 12 for information on the recovery mechanism 1). This means 
that a s ubsequ ent error results in an error reporting error since recovery mechanism 1 splits the 
BERL f BERL lines. 

Figure A-9 describes the contents of the register fields and control bits. 
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Figure A-9: Error Log Register Contents 



Field and Control Bit Descriptions 



ID-Parity Bit 



This field is identical to the ID-Parity field in the error report 
message. 



Location-ID 



This field is identical to the Confinement-ID and Source-ID fields in 
the error report message. 



Error-Type 



This field is identical to the error type field in the error report 
message. The parity bit, which is generated by the BXU, provides 
odd parity over the Error-Type field. 



Valid Bit 



= This setting indicates that the other fields in this register are 
invalid and should be ignored. This bit is set to zero if all error 
messages received in the last error report were invalid. Recov- 
ery does not proceed on error reports that do not have the Valid 
bit set. 



1 = This setting indicates that the other fields in this register are 
valid. 
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Error-Count 



If the Error-Log register is written by software, the Valid bit should 
be set or recovery mechanism 1 is invoked (see Chapter 12). 

This field shows how many errors have occurred since the last time 
the Terminate-Permanent-Error-Window error report was received. 
This 2-bit field is incremented each time an error is reported. The 
counter does not wrap around; if it reaches a value of three, it stops 
until explicitly cleared by the Terminate-Permanent-Error-Window 
error report. This field identifies an overflow condition for the 
software system, as shown in Table A-9. 

This field defaults to a value of 00 B during RESET. 
Table A-9: Interpretation of Error-Count Value 



Error Count 


Definition 


00 


No error information 


01 


A single error report has occurred 


10 


2 error reports have occurred 


11 


3 or more error reports have occurred 



Permanent-Error Bit 



1 = This setting indicates that the error is permanent. Some error 
types are classified as permanent immediately. Others are 
classified as permanent if the same error from the same confine- 
ment area occurs twice in a row within the permanent error 
window. This bit is set by the recovery procedures operating in 
the BXU. 



= This setting indicates that the error is not permanent. 
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Error-Record Register (Address 0F0 H ) 

This register holds the contents of the previous error report to provide additional information to 
software about the error events in the system. Whenever a new error report message is receive the 
contents of the Error-Log register is loaded into the Error-Record register. This register should not 
be written during normal operation because the hardware reads and writes to it. 

The contents of this register are not modified during RESET. In a FRC configuration, this register 
must be written after a cold RESET before being read. Otherwise, the master and checker will 
disagree about the contents of the register since it is not modified during RESET. This register 
contains information about the state of the system before a warm RESET has occurred. 

Figure A- 10 describes the contents of the register fields and control bits. 
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Figure A-10: Error-Record Register Contents 

Field and Control Bit Descriptions 



ID-Parity Bit 
Location-ID 

Error-Type 

Valid Bit 



This bit is identical to the ID-Parity bit in the error report message. 

This field is identical to the Confinement-ID and Source-ID fields in 
the error report message. 

This field is identical to the error type field in the error report 
message. 

= This setting indicates that the other fields in this register are 

invalid and should be ignored. 

1 = This setting indicates that the other fields in this register are 

valid. 
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FRC Register (Address 064 H ) 

The contents of this register determine whether a BXU is part of a master/checker pair and how the 
component responds if it is part of a QMR module. Figure A- 1 1 describes the contents of the register 
control bits. 
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Figure A-11 : FRC Register Contents 



Bit Descriptions 

Master 



1 = This setting indicates either that the agent is the Master of a 
Master/Checker pair, or that there is no Master/Checker pair. 

= This setting indicates that the agent is the Checker. 



Toggle-Master/Checker 



If the COM pin is high at RESET, then this bit is set (this part is a 
master). This read-only bit is not changed by the hardware. 

1 = This setting makes the masters and checkers alternate driving 
the bus on cycle boundaries. 

= This setting makes the masters always drive the bus. 



Access-Checker and Access-Master 

See the QMR register for the usage model for these bits. 
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FRC-Splitting-Control Register (Address 05C H ) 

Writing to this register allows a master/checker pair of BXUs to be split into separate functioning 
components. These register bits are cleared on a cold RESET. They are not modified on a warm 
RESET (except as a result of the FRC Splitting algorithm). Figure A- 12 describes the contents of 
the register bits. 
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Figure A-12: FRC-Splitting-Control Register Contents 



Bit Descriptions 

FRC -Splitting-Disable 



Separated-M/C 



1 = This setting disables FRC splitting logic. 

= This setting instructs the BXU to pursue FRC splitting sequence 

on the next RESET following a permanent error in this module. 

1 = As a result of FRC splitting or an earlier write to this register, 

the BXU sets the Separated-Master I Checker bit to make the 
master/checker pair operate as two independent components. 
Both components act as a master even though one of the 
components does n ot have the M aster bit set. FRC checking on 
the BOUT pin and MODCHK pin is disabled. FRC checking 
remains enabled on all other pins (provides a check on the 
output buffers). 

= The BXU clears this bit to allow checking on all pins used for 
FRC. This master/checker pair does not perform an FRC split. 
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The purpose of this bit is to allow software to configure a board as 
either a pair of components or a single master/ checker uni t. The 
board layout connects the BOUT signal and MODCHK signal 
between the two sets of components. 



Passive 



This bit is manipulated by the fault tolerance logic in the BXU. See 
Chapter 13 for details of splitting algorithms that modify this bit. 



Warm 



My -Permanent-Error 



1 = This setting indi cates th a t the B XU is passive. It cannot drive 
the AP-bus or the BERLj-BERL lines. It can track and perform 
IAC requests sent to its addresses. Normally, a component with 
Passive bit set is the passive component in an FRC split pair of 
components. 

= This setting indicates that the component is active and function- 

ing in normal operation. 

1 = The user controls the Warm bit. By having software set the 

Warm bit, this bit can be used to indicate that the last RESET 
was a warm RESET. The V REF pin is used to distinguish 
between a warm and cold RESET. 

= This bit is cleared after every CO/d RESET. 

1 = This bit is set if the fault tolerance logic is in the process of 

setting the Faulty bit in the FT2 register and the Married bit in 
the QMR register is not set. This bit is set on permanent module 
errors when the module is not part of a QMR unit. This bit is an 
input to the hardware that determines if the module should be 
split on a warm RESET. 



= This setting indicates that the module has not reported a 

permanent module error. 

Write Enable for the My -Permanent-Error Bit 

1 = This setting allows a write operation to the My-Permanent- 

Error bit. 

= This setting does not allow a write operation to alter the My- 
Permanent-Error bit. 
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FT1 Register (Address 054 H ) 

This register in combination with the Fault-Tolerant 2 (FT2) register control the BXU's fault- 
tolerant capabilities. Figure A- 13 describes the contents of the register bits. 
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Figure A-13: FT1 Register Contents 



Bit Descriptions 

Wait-For-BERL 



1 = Settin g this bit to one commands the BXU to wait until the 
BERLj and BERL lines can be checked to guarantee valid data 
from the AP-bus. Data is not used until 0.5 AP-bus clock cycles 
after the next sampling of data. This action adds 1 .5 cycles 
delay per request. 

= This setting indicates that data is used without delay. 



Error-Reporting-Enable 



1 = This setting enables the drivers of the BERL } and BERL pins. 

= This s etting d o es not allow the BXU to send the error message 
on the BERL f BERL lines. 
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Identify -Device-Disable 



COM-Reporting 



1 = This setting ensures proper fault tolerance operation and mini- 
mum latency through the BXU for accesses. This bit must be 
set during normal operation. 

= This setting enables the "Identify Device" IAC. 

This bit must be set to one if fault-tolerant operation is expected. 



1 = This setting instructs the BXU to send an error report on the 
error reporting network when the COM pin is asserted (the 
COM pin is asserted when the COM register is altered using the 
COM register protocol, see Chapter 14). In addition, the COM- 
Register '-Altered bit in the AP -Control register must not be set. 

AP-lnactive 

1 = This setting disables the AP-bus drivers. 

= This setting allows the AP-bus drivers to be enabled. This 

component is functional. 

Write Enable for the AP-lnactive Bit 

1 = This setting allows a write operation to the AP-lnactive bit. 

= This setting does not allow a write operation to alter the AP- 
lnactive bit. 
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FT2 Register (Address 0F4 H ) 

This register holds additional fault-tolerant control parameters that are not contained in the Fault- 
Tolerant 1 (FT1) register. Figure A-14 describes the contents of the register bits. 
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Figure A-14: FT2 Register Contents 



Bit Descriptions 

Backup-Bus-Inactive 



1 = The BXU sets this bit when there is not an active backup AP- 
bus. 



= The BXU clears this bit when there is a backup AP-bus for the 

AP-bus connected to this BXU. 

Write Enable for the Backup-Bus-Inactive Bit 

1 = This setting allows a write operation to the Backup-Bus- 

Inactive bit. 

= This setting does not allow a write operation to alter the 
Backup-Bus-Inactive bit. 
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Force-Correct 1 = If the BXU is in Memory mode, then the Correct pin (COR) is 

always asserted. 



= The COR pin is u nder the control of the BXU (see Chapter 13 
for details on the COR signal and ECC errors). 

Faulty 1 = The BXU sets this bit when the module detects a permanent 

error in its operation and stops operation. 

= The BXU clears this bit when it is in operation. 

Write Enable for the Faulty Bit 

1 = This setting allows a write operation to the Faulty bit. 

= This setting does not allow a write operation to alter the Faulty 

bit. 

Silent-Enable This bit is set before the primary or shadow units of a QMR pair are 

actually married. When the Married bit in the QMR register is 
set, the Silent bit in the QMR register is also set (in both 
modules), but only one module has the Silent-Enable bit set. 
Thus, only one module is silent. 

1 = This setting does not allow the BXU to respond to requests if the 

Silent bit in the QMR register is set. 

= This setting allows the BXU to respond normally. 
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Invalidate-Cache Command (Address 15C H ) 

A write operation to this register location immediately invalidates all lines in the cache directory in 
the BXU. The data word sent with the write request is ignored. This command can only be executed 
safely when the cache is disabled. 
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LBI-Control Register (Address 148 H ) 

This is the major control register for the BXU functions on the L-bus. It is used to set the interleaving 
factor for the cache, to determine whether the BXU should act as an arbitration master on the L-bus, 
and to determine whether the BXU should function in Memory or Processor mode. Figure A- 15 
describes the contents of the register fields and bits. 
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Figure A-15: LBI-Control Register Contents 

Before writing to this register, the Cache-Enable bit in the Cache -Configuration register should be 
cleared and an Invalidate-Cache command sent. This is done because the cache uses the interleaving 
information in this register to control its operation. 

Field and Control Bit Descriptions 



Interleave Mask and Match 



These two fields determine the interleaving factor and matching for 
the cache control logic and for the memory address recognizers in 
the L-bus interface that have interleaving enabled. Table A- 10 
shows the impact of the different configurations of these bits. 



BXU-Mode Bit 



This bit selects the mode of operation of the BXU, as shown i n Table 
A-l 1 This read-only bit is loaded during RESET from the LERLj 
pin. The mode of operation of a BXU can only be set during initiali- 
zation. 
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Table A-10: Interleave-Control Bit Settings for Different Configurations 



Mask Bits 


Match Bits* 


AD 5 , AD 4 Required for Match* 


00 


XX 


XX 


01 


xO 


xO 


01 


x1 


x1 


10t 


Ox 


Ox 


10t 


1x 


1x 


11 


00 


00 


11 


01 


01 


11 


10 


10 


11 


11 


11 



NOTES: 

* "x" designates a "don't care" value; 1 = AD line asserted. 

t The mask bit combination 10 is reached as a result of a bus switch in a four-bus system. Software should 
not use this as a starting configuration. The cache logic handles the 10 combination as a special case 
that is used only under bus switching conditions. This combination is not seen by reading the register. 
Rather, the original setting is read. 



Table A-11: BXU-Mode Settings 



BXU-Mode Bit 


Mode 


LERL 1 at Reset 





Memory 


Asserted (Low) 


1 


Processor 


Deasserted (High) 



Disable-INIT-RAM Bit 



= This setting enables the INIT-RAM memory recognizer. 

1 = This setting disables the INIT-RAM memory recognizer. 
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Arbitration-PBMISBM Bit 



This bit controls the BXU's use of the HOLD/HLDA and HOLDR/ 
HLDAR lines. 



1 = The BXU acts as the primary bus master of the L-bus. This 
provides faster access to the bus when the BXU is operating as 
the primary bus master. 

= The BXU acts as a secondary bus master of the L-bus. 



Arbitration-Enable Bit 



1 = The BXU drives and monitors the arbitration lines in the 
manner selected by the Arbitration-PBMISBM bit. 



= The BXU ignores the arbitration lines. If the BXU acts as a 
primary bus master, it assumes that it has full control of the bus. 
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Local-Bus-Test Register (Address 1 58 H ) 

The BXU utilities the Local-Bus-Test register to allow system diagnostics to check on the type of 
recognition that was done on the previous L-bus request. This register is loaded at the end of each 
L-bus transaction. A read request to this register provides information about the previous request on 
the L-bus. Figure A- 16 describes the contents of the register fields and control bits. 
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Figure A-16: Local-Bus-Test Register Contents 



Bit Descriptions 

Cacheable 



= This setting indicates that the previous request was not cach- 
eable. 



1 = This setting indicates that the previous request was cacheable. 
This means that the CACHE line on the L-bus was asserted, the 
Cache-Inhibit bit was not set in the Match register, the Enable- 
Cache bit was set in the Cache-Configuration register, and the 
request was not a RMW or an I/O prefetch. 
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Partner-Interleave 



= This setting indicates that "my-partner's" bus was not active or 
that the previous request did not have AD 5 and AD 4 set such that 
their value matched the interleaving value for "my-partner's" 
bus. 



1 = This setting indicates that "my-partner's" bus was active at the 
time of the request and that the previous request did have AD 5 
and AD 4 set such that their value matched the interleaving value 
for "my-partner's" bus. 



My-Interleave 



= This setting indicates that this bus WAS NOT ACTIVE or that 
the previous request did not have AD 5 and AD 4 set such that 
their value matched the interleaving value for this BXU. 



1 = This setting indicates that this bus WAS ACTIVE at the time of 
the request and that the previous request did have AD 5 and AD 4 
set such that they matched the interleaving value for this BXU. 



Partner-AP-IAC 



= The previous request WAS NOT AN IAC that flowed onto the 
AP-bus of "my-partner's" BXU. 



1 = The previous request WAS AN IAC that flowed onto the AP- 
bus of my-partner BXU. 



My-AP-IAC 



= The previous request WAS NOT AN IAC that flowed through 
this BXU onto the AP-bus. 



1 = The previous request WAS AN IAC that flowed through this 
BXU onto the AP-bus. 



My-IAC 



= The previous request WAS NOT AN IAC that was handled 
internally by this BXU. 



1 = The previous request WAS AN IAC that was handled internally 
by this BXU. 



Partner's 



= The previous request WAS NOT DIRECTED TO this partner' s 
AP-bus. 



1 = The previous request WAS DIRECTED TO this partner's AP- 
bus. 



Mine 



= The previous request WAS NOT DIRECTED TO this BXU's 
AP-bus. 



1 = The previous request WAS DIRECTED TO this BXU's AP- 
bus. 
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Lock Register (Address 300 H ) 

The Lock register contains the status of the locks used for RMW operations. Figure A- 17 describes 
the contents of the register fields and control bits. 
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Figure A-17: Lock Register Contents 



Field and Control Bit Descriptions 



RMW-Lock-Bits 



Each of these bits is an RMW Lock for a segment of memory 
recognized by this BXU. Table A- 1 2 shows the mapping of AD 6 and 
AD 4 to the RMW -Lock bits in this field. 



Table A-12: RMW Lock Mapping 



RMW-Lock Bit 


Address Bit 


AD 6 


AD 4 











1 





1 


2 


1 





3 


1 


1 



NOTE: 1 == AD line asserted. 

A RMW-Lock bit is set whenever an incoming RMW-Read request 
is accepted by the module, and AD 6 and AD 4 match its address 
segment. 
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A RMW-Lock bit is cleared whenever an incoming RMW-Write 
request is accepted into the module, and AD 6 and AD 4 match its 
address segment. If a lock time-out occurs, the RMW-Lock bit is also 
be reset. Each lock times out independently. The duration of the 
time-out is between 4096 and 8192 clock cycles. 

Whenever an error report is received, all RMW-Lock bits are auto- 
matically set. The timer starts at the beginning of retry (after the end 
of the transient waiting period). This approach allows the locks to 
be set and cleared immediately instead of waiting for the request to 
clear the AP-bus (no possibility of retry). This also provides a way 
to synchronize the timers for primary-shadow operation. 

Write Enable for the RMW-Lock Bits 

1 = This setting allows a write operation to the RMW-Lock bit. 

= This setting does not allow a write operation to alter the RMW- 
Lock bits. 

These bits are protected with a write enable because the BXU 
hardware modifies these bits as a part of normal operation. 

Lock-Timed-Out Bit 1 = This setting indicates that at least one of the f our RMW-Lock bits 

was reset by a time-out. 

= This setting indicates that none of the RMW-Lock bits have been 
reset by a time-out. 

These settings for the RMW-Lock bits indicate a possible corruption 
of the integrity of the memory data. 

Fast-Reply Bit 1 = The external memory controller is operating with fast memory 

components. When data begins to flow back to the BXU, a data 
word is available on eve ry cycle. T he BXU does not assert the 
Reply Deferral signal (RPYDEF) on the AP-bus. The data 
flows directly through the BXU from the L-bus to the AP-bus. 

= The external memory controller is operating with slow memory 
components. Data is not delivered on every clock cycle. The 
BXU asserts RPYDEF on the AP-bus. As data flows back from 
the L-bus, the entire packet is buffered before any data is 
returned on the AP-bus. 

NOTE: 

If the Correct signal (COR) is asserted (either because the Force-Correct bit in the FT2 register is set, or because 
the fault tolerance logic asserted it), data is always fully buffered in the BXU, independent of the state of the 
Fast-Reply bit. This allows for the slower timing required by the external ECC circuit when it is performing 
single bit corrections. 
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Logical-ID Register (Address 044 H ) 
Logical-ID Register (local) - (Address 144 H ) 

This register holds the logical identification for the BXU. In every case, all BXUs in the same module 
share the same logical identification value. 

The register address accesses two registers: the Logical-ID register located in the AP-bus interface 
and the Logical-ID(local) register located in the L-bus interface. All write operations to register 
address 044 H (including the Identify Device Order) modify both registers. Read operations to register 
address 044 H read the register in the AP-bus interface. Read and write operations to register address 
144 H use only the Logical-ID( local) register in the L-bus interface. Access to the Logical-ID( local) 
register using register address 144 H is primarily made available for test purposes. There is no need 
to access this register during normal operations. Figure A- 18 describes the contents of the register 
fields. 
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Figure A-18: Logical-ID Register Contents 



Field Descriptions 

Unit-ID 



This field holds the logical identification for the BXU. 
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Mask and Match Registers (Addresses 160 H through 174 H ) 

The contents of the Match register determine the value of the L-bus address that is recognized by the 
BXU. This register provides a base address for a partition of memory recognized by the BXU. The 
contents of the Mask register determine whether certain bits should be ignored during the address 
recognition. 

The Mask register should always be setup before the Match register. This avoids matching on 
incorrect address patterns. 

Before writing to these registers, the Enable-Cache bit in the Cache-Configuration register should 
be cleared and an Invalidate-Cache command sent. This is performed because the cache uses the 
information in this register to control its operation. 

This description describes the three sets of identical Mask and Match registers shown in Table A- 1 3. 

Table A-13: Address for Match and Mask Registers 



Register Address 


Register 


160„ 


Matcho 


164 H 


Mask 


168 H 


Match, 


16C H 


Mask, 


170 h 


Match 2 


174 h 


Mask 2 



Figure A- 19 describes the contents of the register fields and control bits. 
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Figure A-19: L-Bus Mask and Match Register Contents 
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Field Descriptions 

Function 00 = This setting disables the Mask and Match registers. 

01 = This setting enables the bus recovery of the Mask and Match 
registers. The Mask and Match registers use the interleave 
configuration bits in the LBI-Control register (which has the 
option of no interleaving). The cache control logic also uses the 
interleaving bits to control its operation. 

10 = This setting disables interleaving. LAD 5 and LAD 4 are ignored 

during address matching. If the cache is being used, thelgnore- 
Interleave -Control bit in the Cache-Configuration register 
must be set. 

1 1 = This setting disables the interleaving. LAD 5 and LAD 4 are 

ignored during address matching. This recognizer is a backup. 
Requests that use this recognizer are placed in the BXU's 
internal partner queue to enable tracking of the active bus (i.e., 
the bus where the BXUs have the Function bits set to 10). It 
only becomes active upon the permanent failure of its partner's 
bus. If the cache is being used, the 1 gnore-Interleave-Control 
bit in the Cache-Configuration register must be set. 

The Mask and Match registers operation is also conditional on the 
Faulty bit in the FT2 register and the Inactive bit in the FT1 register. 
If the Faulty bit is set, all of the Mask and Match registers of the L- 
bus interface are disabled. If the Inactive bit is set in the BXU on 
this bus, then all requests go to the backup bus. If the Inactive bit is 
set in the BXU on the backup bus, then all requests go to this bus. 

Sequence 1 = Any write request that is processed by this recognizer causes the 

BXU to insert additional wait cycles on the L-bus until the 
request is completed on the AP-bus. Internal prefetch requests 
are also held back until the access is completed. This is required 
to maintain access ordering between multiple requests to loca- 
tions that may deadlock, and thus, force the receiver to send a 
reissue reply back to the sender. Any address range that could 
encounter this deadlock condition must set the Sequence bit. 
See "System Configurations" section of Chapter 8 for more 
information. 

= No additional wait states are added to L-bus write requests. 



A-46 



inteT 



APPENDIX A 



Cache-Inhibit 



This bit is for use during cache diagnostics. During normal operation 
it should always be a zero. For more information on the use of this 
bit see the "Diagnostic Support Functions" section of Chapter 8. 



1 = Requests that are processed by this recognizer are not cached 
(even if the Cacheable bit is asserted). This bit overrides all 
other cache control bits. 

= Requests that are processed by this recognizer may go to the 
cache if the Function bits (see below) and other cache control 
bits are set correctly. 



Recognize 



In the Mask registers, each bit in this field that is set causes the L-bus 
address bit to be compared against the corresponding Match register 
bit. If a bit is cleared, then that bit position is considered "don't care" 
during address recognition. 



In the Match registers, each bit in this field is compared against the 
corresponding bits in the L-bus address cycle. Thus, these bits 
provide abase address for the partition of memory that is recognized 
by this address recognizer. 
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Maxtime Register (Address 058 H ) 

The value in this register determines the length of time that the BXUs remain quiescent following 
the beginning of an error report. Figure A-20 describes the contents of the register bits. 
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Figure A-20: Maxtime Register Contents 

Field and Control Bit Descriptions 



Max-Time 



Maxtime-Test-Enable 



These four bits adjust the transient waiting period from 1 6 microsec- 
onds to 0.5 seconds with a 16 MHz Bus Cycle (CLK2 = 32 Mhz). 
The following formula shows how to increase the time period as a 
function of the clock frequency: 

Number of cycles =3 + 2 8 + M *2 12 + M 1 *2 16 + M 2 *2 20 + M 3 *2 24 

1 = This setting allows testing of the Maxtime counter. 

= This setting inhibits the testing of the Maxtime counter (normal 
setting for system operation). 



Write Enable for the Maxtime-Test-Enable Bit 

1 = This setting allows a write operation to the Maxtime-Test- 
Enable bit. 

= This setting does not allow a write operation to alter the 
Maxtime-Test-Enable bit. 
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Module-Error-ID Register (Address 0E4 H ) 

This register provides the Location-ID field for error reports that originate in a module confinement 
area. The Location-ID field consists of the Source-ID and the Confinement-ID fields of this register. 
Figure A-21 describes the data fields and the control bits of this register. 
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Figure A-21 : Module-Error-ID Register Contents 



Field and Control Bit Descriptions 



ID-Parity Bit 



This bit provides even parity for the Source-ID and Confine- 
ment-ID fields in this register. When software loads new values 
into these fields, it must also load the ID -Parity bit to ensure that 
there is an even number of bits with the value of one in this 16- 
bit register. 



Source-ID 



This 7-bit field uniquely identifies a BXU within a confinement 
area. Software must assign a value to this field. Software uses 
the Source-ID field in the Error-Log register to specifically 
identify which BXU (or Master/Checker pair) had issued the 
error report message. 



Confinement-ID 



This 8 -bit field uniquely identifies a confinement area within the 
system. All Confinement-ID values in the system must be in the 
range 4 to 255 (the values of zero to three are reserved for the 
bus confinement areas). The BXUs use this value in the error 
report messages to determine which recovery actions to take. 
Software must assign a value to this field. The Confinement-ID 
field is used when the BXU reports an error in its module 
confinement area. 
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Physical-ID Register (Address 040 H ) 
Physical-ID Register(local) - (Address 140 H ) 

This register contains a unique identifier for a specific BXU on an AP-bus. The register address 
accesses two registers: the Physical-ID register located in the AP-bus interface and the Physical- 
ID(local) register located in the L-bus interface. All write operations to register address 040 H 
(including the Identify Device Order) modify both registers. Read operations to register address 
040 H read the register in the AP-bus interface. Read and write operations to register address 140 H 
use only the Physical-ID (local) register in the L-bus interface. Access to Physical-ID (local) using 
register address 140 H is primarily made available for test purposes. There is no need to access this 
register during normal operations. Figure A-22 describes the contents of the register fields and bits. 
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Figure A-22: Physical-ID Register Contents 



Field and Control Bit Descriptions 



Component-ID 



The Component-ID field allows unique identification of com- 
ponents with the same Class-ID field (see following field dis- 
cussion) on the same bus. The default value of this field is 00 H . 
This register can be loaded with a unique value by the Identify 
Device Order, or by performing a write operation. 



Class-ID Bit 



This bit must be set to a value of zero during the initialization 
routine. The value of one is reserved for future product prolif- 
erations. 
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Prefetch-Control Register (Address 240 H ) 

The prefetch unit of the BXU uses the Prefetch-Control register for control. Figure A-23 describes 
the contents of the control bits. 
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Figure A-23: Prefetch-Control Register Contents 



Control Bit Descriptions 



L 'O-ChanneL- Active 



This read-only bit is automatically set whenever a Start-Channel 
command is performed after the Enable bit is set. This bit is 
automatically cleared whenever a bus switch (attach or detach) 
occurs that involves this BXU (either its bus or its partner's bus). 



1 = This setting indicates that Channel Q of the prefetch unit is active. 
The channel actively monitors L-bus traffic for address matches 
to perform the associated prefetch work. 



II O -Channel f Active 



= This setting indicates that Channel is inactive. 

This read-only bit is automatically set whenever a Start-Channelj 
command is performed after the Enable bit is set. This bit is 
automatically cleared whenever a bus switch (attach or detach) 
occurs that involves this BXU (either its bus or its partner's bus). 



1 = This setting indicates that Channel is active. The channel 
actively monitors the L-bus traffic for address matches to 
perform the associated prefetch work. 

= This setting indicates that Channel is inactive. 
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Enable The Enable bit is automatically cleared whenever a bus switch 

(attach or detach) occurs that involves this BXU (either its bus or its 
partner's bus). This means that recovery software needs to explicitly 
turn the prefetch function back on after the bus switch, if appropri- 
ate. 

= This setting disables the prefetch unit. 

1 = This setting makes both prefetch channels available for use in 

transferring sequential I/O data streams. 
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Primary-Catastrophe Command (Address 108 H ) 

A write operation to this register location causes the BXU to generate a Primary -Catastrophe error 
report on the serial error reporting network. The data word sent with the write request is ignored. 

WARNING: The Testing-Enable bit in the Test-Detection register must be set before executing this 
command. Before setting the Testing-Enable bit, there should be a single bit with the value of one 
in the COM register. 
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Private Memory Match and Mask Registers (Add. 178 H , 17C H ) 

The private memory Mask and Match registers provide the ability to support SRAM on the L-bus as 
a normal memory, instead of cache. The Mask register should always be setup before the Match 
register. This is required to avoid matching on incorrect address patterns. 

When a match occurs on the privat e mem ory recognizer, the WYj pin is asserted and the address bit 
from the LAD 14 pin is driven on the WY pin. If the address range of the private memory recognizer 
overlaps the range of an AP-bus recognizer (160 H -174 H ), then the request is considered to be to the 
private memory. 

Figure A-24 describes the contents of the register fields and control bits. 
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Figure A-24: Private Memory Mask and Match Register Contents 



Field and Control Bit Descriptions 



Recognize 



In the Mask registers, each bit in this field that is set causes the 
corresponding L-bus address bit to be compared against the 
corresponding Match register bit. If a bit is cleared, then that bit 
position is considered a "don't care" during address recognition. 



In the Match registers, each bit in this field is compared against the 
corresponding bits in L-bus address cycles. Thus, these bits provide 
a base address for the partition of memory that is recognized by this 
address recognizer. 



Enable 



= This setting disables the Mask and Match registers. 



1 = This setting enables the Mask and Match registers. The opera- 
tion of the Mask and Match registers is also conditional on the 
Faulty bit in the FT2 register (module failure). If the Faulty bit 
is set, then the private memory Mask and Match registers are 
disabled. 
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Processor-Priority Registers (Address 000 H and 020 H ) 

The register with address 000 H (Processor-Priority ) holds the priority of the task (process) which 
processor on the BXU's L-bus is currently executing. Similarly, the register with address 020 H 
(Processor-Priority j ) holds the priority of the task (process) which processor on the BXU's L-bus 
is currently executing. Figure A-25 describes the contents of the register fields and bits. 
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Figure A-25: Processor-Priority Register Contents 



Field and Control Bit Descriptions 



Message-Buffer-Full Bit 



This read-only bit shows the state of the IAC pin for processor^ or 
processor . The state of the IAC pin may be changed by changing 
the value of the Message-Data-Valid bit. 



1 = This setting indicates that an IAC message is waiting for the 
80960MC processor. Any subsequent IAC message requests 
received fo r this processor are sent the No Acknowledgement 
reply. The IAC pin is asserted (low). 

= T his setting indicates that the IAC buffer is empty. The IAC pin 
is not asserted (high). 



Message-Data-Valid Bit 



1 = This setting indicates that the BXU, whic h set the Message- 
Buffer-Full bit, has the IAC message data. IAC pin for proces- 
sor^ or processor is asserted. This bit is only set after the reply 
to the message is sent on the AP-bus. When the Message-Data- 
Valid bit is set, the Message-Buffer-Full bit is also set. The 
Message-Buffer-Fullbit is set in all BXUs in the module, but the 
Message-Data-Valid bit is only set in a single BXU. 



= This setting indicates that the message buffer does not hold 
valid data. 
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Priority These five bits show the priority of the activity that the processor is 

currently executing. Priority is the lowest priority and 3 1 is the 
highest. 

Write Enable for Message-Data-Valid Bit 

1 = This setting allows a write operation to the Message-Data- 
Valid bit. 

0= This setting does not allow a write operation to alter the 
Message-Data-Valid bit. 

Write Enable for the Priority Bit 

1 = This setting allows a write operation to the Priority bits. 

= This setting does not allow a write operation to alter the Priority 
bits. 
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QMR Register (Address 0E0 H ) 

The contents of this register determine whether a module is part of a QMR pair, and whether it should 
function as the primary or shadow unit Figure A-26 describes the contents of the register fields. 
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Control Bit Descriptions 



Figure A-26: QMR Register Contents 



Silent 



1 = This setting does not allow the BXU to reply to bus requests if 
the Silent-Enable bit in the FT2 register is set. In other words, 
the Silent bit selects which module does not drive the AP-bus. 
This bit is automatically set by a Sync-Refresh command that 
may be issued to this BXU or its partner. When the Silent bit is 
set, the Toggle-Primary I Shadow bit must be cleared. The 
Silent bit has the same value in both the primary and shadow 
units. The Silent bit is also used to control the precise time at 
which silent operation begins. 



Married 



= Normal operation. 

1 = This setting indicates that the FRC pair is married to another 

FRC pair. 

= This setting indicates that the FRC pair is not married. 
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Write Enable for the Married bit 

1 = This setting allows a write operation to the Married bit. 

= This setting does not allow a write operation to alter the Married 
bit. 



Toggle -Primary I Shadow 



1 = This setting forces the primary and shadow units to alternate 
driving the bus every other bus cycle. 



= This setting allows only the primary unit to drive the bus. 

If Toggle-Primary I Shadow bit is set, the bus drive sequence is 
Primary-Master, Primary-Checker, Shadow-Master, Shadow- 
Checker (see Table A- 15 for more specific information). The 
Toggle-Master I Checker bit in the FRC register must not be set if 
Toggle-Primary/Shadow is set. Conversely, The Toggle-Primary/ 
Shadow must not be set if Toggle-Master/Checker bit in the FRC 
register is set. 



Shadow 



1 = This setting indicates that this BXU is part of a shadow 
module. 



= This setting indicates that this BXU is part of a primary module, 

or that module shadowing is not used. 

This information determines whether this BXU drives the AP-bus 
and responds to an IAC request that has the Access bit set in the IAC 
address. 

Write Enable for the Shadow Bit 

1 = This setting allows a write operation to the Shadow bit. 

= This setting does not allow a write operation to alter the Shadow 
bit. 

Access-Shadow and Access-Primary 

The value of these bits determines which BXU in a QMR configu- 
ration responds to an IAC request using a logical address (IAC 
access type 0010 B ) when the Access bit is set. In addition, this bit 
determines which BXU drives the AP-bus in the QMR module. 

The Married, the Shadow, the Access-Shadow, the Access-Primary, 
the Access-Master and the Access-Checker bits are used to deter- 
mine which BXU responds to an IAC request type 0010 B when the 
Access bit is set, as shown in Table A- 14. Note that requests using 
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the Access bit must originate from the AP-bus. The registers that are 
not part of the AP-bus interface logic ignore the Access bit, and all 
BXUs where the appropriate portion of the address matches the 
Unit-ID field in the Logical-ID register perform the request. 

Table A-14:Determination of Which BXU Performs an IAC Type 0010 



Configuration 


BXU' that 
Performs 
Request 


Conditions 


Access-Primary 
Bit in QMR 
Register 


Access-Shadow 
Bit in QMR 
Register 


Access-Master 
Bit in FRC 
Register 


Access-Checker 
Bit in FRC 
Register 


Married Bit 
mQMR 
Register 


Shadow Bit 
in QMR 
Register 


QMR 


Shadow Checker 


- 


1 


- 


1 


1 


1 


Shadow Master 


- 


1 


1 


- 


1 


1 


Primary Checker 


1 


- 


- 


1 


1 





Primary Master 


1 


- 


1 


- 


1 





FRC 


Checker 


1 








1 








Master 


1 





1 












NOTE: 

* The BXU is identified by the Master bit in the FRC register and the Shadow bit in the QMR register. For example, if the Shadow bit and the 
Master bit are set, the BXU is a Shadow Master. 



The determination of which BXU replies to the requests in a QMR 
or FRC configuration is listed in Table A- 15. No other bits 
determine which BXU replies. Note that Table A-15 applies to all 
requests. This means that IAC requests using a physical address are 
not allowed while toggling is enabled. Even though multiple 
components may perform the request, only one component replies 
on the bus. 
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Table A-15: Determination of Which BXU Replies to Requests 



Configuration 


BXU 1 that Replies 


Conditions 


Silent Bit In 
QMR Register 


Married Bit in 
QMR Register 


Toggle-Primary /Shadow 
Bit in QMR Register 


Toggle-Master /Checker 
Bit in FRC Register 


FRC 


Master (M) 








- 





Toggle M, C, M, C, etc. 











1 


QMR 


Primary Master (PM) 





1 








Toggle PM, PC 2 , PM, etc. 





1 





1 


Toggle PM, PC, SM 3 , SC, 
PM, etc. 





1 


1 





All 


No BXU Replies 


1 4 


- 





- 



NOTES: 

1 . The BXU is identified by the Master bit in the FRC register and the Shadow bit in the QMR register. For example, if the Shadow bit and the 
Master bit are set, the BXU is a shadow master. 

2. PC is an abbreviation for Primary Checker. 

3. SM is an abbreviation for Shadow Master; SC is an abbreviation for Shadow Checker. 

4. The Silent-Enable bit in the FT2 register is also set. 
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Shadow-Catastrophe Command (Address 10C H ) 

A write operation to this register location causes the BXU to generate a Shadow-Catastrophe error 
report on the serial error reporting network. The data word sent with the write request is ignored. 

WARNING: The Testing-Enable bit in the Test-Detection register must be set before executing this 
command. Before setting the Testing-Enable bit, there should be a single bit with the value of one 
in the COM register. 
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Spouse-ID Register (Address 0C4 H ) 

This register is used by BXUs in a QMR module configuration. For a married Primary/Shadow 
pair, this register holds the identification information of its "mate". Figure A-27 describes the 
contents of the register fields. 
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Figure A-27: Spouse-ID Register Contents 



Field Descriptions 

Spouse-ID 



This 8-bit field corresponds to the Confinement-ID field in the 
Module-ID register of the BXUs to which the module is married. 
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Synchronize-Refresh Command (Address 1 1 C H ) 

A write operation to this register location address causes the BXU to generate the Sync-Refresh 
error report on the serial error reporting network. The data word sent with the write request is ignored. 
The BXUs in Memory mode receiving the Sync-Refresh error report perform the following 
actions: 

• Assert the Force Refresh (FRF) pin for one cycle at the beginning of the transient waiting 
period. 

• Set the Enable bit in the AP -Match register to turn on AP-bus address recognizer. This is 
done by means of an error report because it is the only time that the memory module can be 
guaranteed to be idle. 

• Set the Silent bit in the QMR register. This action causes the module that previously had the 
Silent-Enable bit set in the FT2 register to become silent (it will not reply to any accesses). 

WARNING: The Testing-Enable bit in the Test-Detection register must be set before executing this 
command. Before setting the Testing-Enable bit, there should be a single bit with the value of one 
in the COM register. 
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System-Bus-ID Register (Address 14C H ) 

This register uniquely identifies the BXU as attached to one of the four AP-buses. Figure A-28 
describes the contents of the register bits. 
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FIELD 



RESET DEFAULT 
VALUES 



AP-BUS-ID 



* THIS BIT IS SET TO A VALUE OF ONE IF MODCHK IS ASSERTED, 
OTHERWISE IT IS SET TO ZERO. 

t THIS BIT IS SET TO A VALUE OF ONE IF BOOT IS ASSERTED, 
OTHERWISE IT IS SET TO A VALUE OF ZERO. 
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Figure A-28: System-Bus-ID Register Contents 



Field Description 

AP-Bus-ID 



These two bits uniquely identify the BXU as attached to one of up 
to four buses. At RESET, if MODCHK and BOUT are pulled high 
(not asserted) by external resisters, the AP-Bus-ID field is set at a 
value of 00. 
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Terminate-Permanent-Error-Window Command (Address 110 H ) 

A write operation to this register causes the BXU to generate a Terminate-Permanent-Err or '-Window 
error report on the serial error reporting network. The data word sent with the write request is ignored. 
This action closes the permanent error window, so that a reoccurrence of a previous error is not 
recorded as permanent. 

WARNING: The Testing-Enable bit in the Test-Detection register must be set before executing this 
command. Before setting the Testing-Enable bit, there should be a single bit with a value of one in 
the COM register. 
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Test-Detection Register (Address 060 H ) 

Figure A-29 describes the contents of the register fields and bits 
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Figure A-29: Test-Detection Register Contents 



Field and Control Bit Descriptions 



Parity-Test 



1 = This setting instructs the BXU to test the parity checking logic. 
Before setting this bit, there must be a single value of one in the 
COM register. This must be done to prevent a FRC error from 
occurring as a result of an incorrect FRC test. (The FRC test 
occurs automatically as soon as the Testing-Enable bit is set.) 

= This setting causes the BXU not to test the parity checking 
logic. 

Normal operation checks that the parity tree is working correctly. If 
there was a failure in the parity tree, then the BXU signals a parity 
error because its computed parity differs from that generated by the 
sending component. However, normal operation does not check the 
logic that compares the check bit with the result computed from the 
internal parity tree. Moreover, neither the inputs nor the output are 
checked during normal operation because there normally is parity 
agreement. The parity testing bits allow testing of this logic. 
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The Parity-Test bits invert the CHK pin(s) sampled from the bus. 
This action creates a parity error. The parity comparing logic is 
expected to detect the error. The component reports an error if the 
Parity-Test bit is set and a parity error is detected. For a particular 
parity tree to be tested, the appropriate Parity-Test bit must be set. 
Table A- 1 6 shows the mapping of Parity -Test bit to which part of the 
tree is corrupted. 



Parity testing can be done on-line, but a real parity error (i.e., non- 
forced) during the test may go undetected. The success of the test 
is noted by checking the Error-Log register after the test. 

Table A-16: Parity-Test Bits Versus Parity Tree Corrupted 



Parity Test Bits 


Function 


F1 


FO 


X 


1 


CHK parity error is forced 


1 


X 


CHK! parity error is forced 


1 


1 


Illegal 



Testing-Enable Bit 



1 = This setting enables the BXU to execute the testing commands. 



= This setting disables the BXU from executing the testing com- 

mands. 

This bit indicates a test function is enabled. When this bit has a value 
of one, then FRC testing is automatically enabled. For correct 
operation, the COM register must have a single bit with a value of 
one while the Testing-Enable bit is set. This bit is reset by any error 
report. The resetting of this bit by the error report at the end of a test 
ensures that the test is not retried in the event that another error report 
is generated as a result of the test. This eliminates the possibility of 
any infinite loops. This bit is also cleared when the COM register is 
altered using the COM register protocol (see Chapter 14 for details 
on the protocol). This bit defaults to a value of zero. 

Write Enable for the Testing-Enable Bit 

1 = This setting allows a write operation to the Testing-Enable bit. 

Write protection is provided because access to this register is 
retried when the test results in an error report. 

= This setting does not allow a write operation to alter the Testing- 
Enable bit. 
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Test-Report Command (Address 104 H ) 

The Test-Report command forces the BXU to test the error reporting network. The type of error 
report generated is determined by the contents of the Test-Type register. A write operation to this 
register location causes the BXU to execute a Test-Report command. The data word sent with the 
write request is ignored. 

WARNING: The Testing-Enable bit in the Test-Detection register must be set before executing this 
command. Before setting the Testing-Enable bit, there should be a single bit with a value of one in 
the COM register. 
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Test-Type Register (Address 0C0 H ) 

The Test-Type register is used in conjunction with the Test-Report command. The Test-Report 
command forces the BXU to test the error reporting network. The Test-Type register determines the 
type of error report that is generated. Figure A-30 describes the contents of the register field. 
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Field Descriptions 



Figure A-30: Test-Type Register 



Test-Type 



This 20-bit field determines the type of error that is generated. Each 
test bit in this register specifies a specific error type, as shown in 
Table A- 17. Multiple errors may be loaded to test the priority 
circuits of the BXU. Note that the contents of this field controls the 
test types generated. The actual test occurs as a result of a Test- 
Report command. The specified tests are performed if the Testing- 
Enable bit in the Test-Detection register is set. 



If all the bits in the Test-Type field are set to a value of zero and a Test- 
Report command is issued, then the result of the command is no 
operation and no error report is generated. This action prevents 
sending another error report when the Test-Report command is 
retried at the end of the reporting and recovery cycle. If the Test- 
Detection register is not enabled and the Test-Report command is 
issued, then the result of the command is no operation and no error 
report is generated. This action also prevents sending another error 
report because the error report causes the Test-Detection register to 
reset the Testing-Enable bit in all BXUs. 

NOTE 

In order to insure software compatibility with members of the 960 product family RESERVED error condition 
bits must remain at their reset default values of zero. 
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Table A-17: Settings for Test-Type Field 


Bit Number 


Type of Error 


Comment 


19 


Unsafe-Confinement-Area (Bus) 




Permanent error on the BERL pins 


18 


Reserved 


Reserved for use by future AP-bus 
components 


17 


Unsafe-Confinement-Area 
(Module) 


Permanent error on the LERL pins 


16 


Unsafe-Confinement-Area 
(Module) 




FRC error on the MODCHK pins 


15 


Unsafe-Confinement-Area 
(Module) 


ERR pin asserted 


14 


Primary-Catastrophe 




13 


Shadow-Catastrophe 




12 


Error-Reporting-Error (Bus) 




Transient error on the BERL pins 


11 


Error-Reporting-Error (Module) 


Transient error on the LERL pins 


10 


Bus-Arbitration 




9 


Bus-Parity 




8 


Component 




7 


Reserved 


Reserved for use by future AP-bus 
components 


6 


Uncorrectable-Array-Error 




5 


Correctable-ECC 




4 


Corn-Altered 




3 


Attach-Bus 




2 


Detach-Bus 




1 


Terminate-Permanent-Error- 
Window 







Sync-Refresh 





NOTE: In order to ensure software compatibility with future members of the 960 product family reserved 
error condition bits must remain at their reset default values of zero. 
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A 

Access Bit 8-14, 12-16 

Active Module 8-45 

Active/Passive Module 8-46 

AD 3t -AD (Address/Data) Lines on the AP-Bus 

definition 7-3 

usedinFRC 11-4 

used with request and reply packets 7-11 

Address Block for Cache 8-16 

Address Decoder 

I/O interface 5-2 

memory interface 4-2 

Address Latch/Demultiplexer 

I/O interface 5-3 

memory interface 4-2 

ADS (Address Status) Signal 

definition 3-4 

timing diagram 3-9 

used by the 82786 interface 5-13 

used by the burst logic 4-3 

used by the timing control logic 5-3 

ALE (Address Latch Enable) Signal 

definition 3-4 

timing diagram 3-8 

used by an address latch/demultiplexer 4-2, 5-3 

used by the byte enable latch 4-6 

used by the SRAM interface logic 4-6 

used in the SRAM interface logic 4-9 

AP-Bus 

communication between buses 13-22 

confinement area 11-1 

definition 7-1 

protocol 7-24 

recovery 13-18 

signal groups 7-3 

signal timing 7-32 

time-outs 11-2 

topology 7-2 

AP-Bus Interface Logic on the BXU 

bus interleaving 8-10 

functions, list of 8-3 

IAC address recognition (Identify Device Order) 8-31 

IAC address recognition (Logical) 8-30 

IAC address recognition (Message) .8-31, 8-32 

IAC address recognition (Physical) 8-30 
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memory address recognition 8-4 

AP-Control Register 

definition A-8 

used during initialization 14-5 

used during the parameterization phase of initialization 14-7 

used for bus switching 13-17 

used in an error type 12-13 

AP-Mask Register 

definition A- 10 

example of use during identification phase of initialization 14-24, 14-27 

interleaving 8-7 

used for address recognition on the AP-bus 8-4 

used for marrying modules by a special agent 14-12 

used for processor module recovery 13-10 

used to marry processor modules using another module 13-11 

AP -Match Register 

definition A- 10 

example of use during identification phase of initialization 14-24, 14-27 

interleaving 8-8 

used for address recognition on the AP-bus 8-4 

used for marrying modules by a special agent 14-11 

used to marry processor modules using another module 13-10 

ARB 3 -ARB (Arbitration) Signals 

definition 7-4 

signal duplication in fault-tolerant systems 1 1-2 

timing 7-32 

usedinFRC 11-5 

Arbitration 

AP-bus example 7-26 

L-bus example 3-18 

protocol for the AP-bus 7-24 

protocol for the L-bus 3-17 

timing on the L-bus , 3-18 

Arbitration-ID Register 

definition A-12 

example of use during identification phase of initialization 14-20 

example of use for arbitration on the AP-bus 7-26 

used during the identification phase of initialization 14-6 

used for arbitration on the AP-bus ...7-25 

used for marrying modules by a special agent 14-12 

used to marry processor modules using another module 13-9 

Asynchronous I/O System 15-2 

Attach-Bus Error Type 

definition 12-12 

determining which bus failed 13-18 

used after a bus switch 13-21 

Attach-Bus Command 

definition A-14 

description 13-21 

used after a bus switch 13-21 

used to issue an error report 12-12 
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B 

Bad-Access Reply 

definition 7-16 

usage 8-14 

BADAC (Bad Access) Signal 3-30 

used by cache logic of BXU 8-25 

used with IAC requests 8-14 

used with write requests on the AP-bus 8-39 

BE 3 -BE (Byte Enable) Signals 

definition 3-4 

timing 3-5 

timing diagram 3-5 

used by the BXU and cache 8-19 

used by the byte enable latch 4-5 

used by the DRAM controller 4-15 

used by the prefetch unit of the BXU 8-35 

BERL f BERL (Bus Error) Signals 

definition 7-4 

diagnostic tests 12-15 

timing 7-32 

timing with error message 12-2 

used by the error reporting network 12-1 

used during phase one of error reporting sequence 12-5 

used during phase two of error reporting sequence 12-7 

usedinFRC 11-4 

BOUT (Bus Output Control) Signal 

definition 7-5 

used as a detection mechanism in fault-tolerant systems 1 1-4 

used during initialization 14-3 

usedinFRC 11-4 

Burst Logic 

memory interface 4-3 

signal flow 4-4 

Burst Transaction 3-11 

Bus Recovery 13-18 

Bus Switching 

communications after a bus switch 13-22 

effects on cache accesses 13-25 

effects on I/O prefetch accesses 13-26 

effects on IAC operations 13-20 

effects on memory accesses 13-25 

effects on memory range recognition 13-23 

overview 13-16 

recovery 13-17 

Bus-Arbitration Error Type 

definition 12-11 

determining which bus failed 13-18 

Bus-Error-ID Register 

definition A- 15 

example of use during a processor 

module marriage 14-30 

example of use during identification 
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phase of initialization 14-22,14-23,14-25,14-26 

example of use during parameterization phase of initialization 14-19 

used as the location in an error message 12-10,12-11,12-12 

used during initialization 14-5 

used for bus recovery 13-18 

used for marrying modules by a special agent 14-12 

used in an error message 12-9,12-10,12-1 1 

used to marry processor modules using another module 13-10 

used with the attach-bus sequence 13-21 

Bus-Parity Error Type 

definition 12-12 

determining which bus failed 13-18 

Byte Marks 7-13 

c 

Cache Directory and Control Logic of the BXU 

coherency 8-15,8-18,8-19, 

configuration and control 8-18 

directory 8-15 

fill sequence 8-25 

memory structure 8-15 

operation 8-15 

operation with multiple BXUs , 8-19 

reaction to bad-access reply 8-25 

SRAM Interface 8-20 

SRAM support logic 8-21 

update policy 8-18 

Cache Memory 

attached to the BXU 8-15 

retry operation 13-5 

CACHE Signal 3-6 

used by a BXU in Memory mode 8-25 

used by a BXU in Processor mode 8-15 

used by the prefetch unit of the BXU 8-35 

Cache-Configuration Register 

cache configuration and control 8-18 

considerations after a bus switch 13-25 

definition A-17 

directory 4 8-17 

effect on replacement algorithm 8-15,8-16 

selecting the timing mode 8-12 

slow cache timing operation 8-12 

timing options 8-42 

used for INIT-RAM address recognition on the L-bus 8-11 

used for testing the cache directory 8-39 

Cache-Test Register 

definition , A-21 

testing directory 8-39 

CHK,-CHK (Check) Signals 

definition 7-4 

timing 7-33 

used as a detection mechanism in fault-tolerant systems 1 1-2 
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usedinFRC 11-4 

CLK2 (Processor Clock) or CLK (Bus Clock) 

AP-bus signal 7-4 

CLK2 requirements 3-12 

generation 3-12 

relationship of CLK2 and CLK 3-7 

Coherency 8-15,8-18,8-19 

Cold Start 14-9 

COM (Communication) Signal 

definition 7-6 

used by a memory controller 13-14 

used during initialization 14-3 

used to indicate an error 12-15 

COM Register 

definition A-23 

used by a memory controller interface 13-14 

used during the parameterization phase of initialization 14-7,14-8 

COM Protocol 14-7 

COM- Altered Error Type 

definition 12-12 

serial COM protocol 14-7,14-8 

Commands Listed in Appendix A 

Attach-Bus A-14 

Detach-Bus A-25 

Invalidate-Cache A-37 

Primary -Catastrophe A-53 

Shadow-Catastrophe A-61 

Sync-Refresh A-63 

Terminate -Permanent-Error -Window A-65 

Test-Report A-68 

See also individual commands 

Comparators for I/O System 15-5 

Component Error Type 12-12 

Component-Specifier Register 

definition A-24 

Configurations With the BXU 8-44 

Confinement 

AP-bus 11-1 

definition 10-6,11-1 

example of operation 11-6 

module area 11-3 

Confinement-ID Field 12-9 

COR (Correct) Signal 13-13 

Correctable- ECC Error Type 12-12 

CR (Cache Read) Signal 8-20 

CW (Cache Write) Signal 8-20 

Cycle-In-Progress Signal 

definition 4-3 

used by the burst logic 4-3 

used by the DRAM arbiter 4-12 

used by the DRAM timing and control 4-13,4-14 

used by the timing control logic 4-5 
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D 

Data Transceivers 

I/O interface 5-1 

memory interface 4-1 

Default Initialization Phase 

definition 14-1 

pins sampled 14-3 

DEN (Data Enable) Signal 

definition 3-4 

timing diagram 3-9 

used by a data transceiver 4-1,5-3 

used by the burst logic 4-3 

Detach-Bus Error Type 

definition 12-12 

determining which bus failed 13-18 

used after a bus switch 13-18 

Detach-Bus Command 

definition A-25 

used after a bus switch 13-18 

used to issue an error report 12-12 

Detection Mechanism 

definition 10-7,11-1 

FRC 11-3 

parity 11-2 

signal duplication 11-2 

time-outs 11-2 

DRAM Address Multiplexer 4-11 

DRAM Arbiter 4-12 

DRAM Controller 4-10 

DRAM Interleaving 4-17 

DRAM Refresh Interval Timer 4-11 

DRAM Timing and Control 4-12 

DRAM Timing Considerations 4-15 

DT/R (Data Transmit/Receive)Signal 

definition 3-4 

timing diagram 3-10 

used by a data transceiver 4-1,5-3 

used by the 82586 interface 5-10 

E 

ECC (ECC Error) Signal 13-13 

Error Message 12-6 

Error Report Log 12-14 

Error-Log Register 

defining a permanent error 13-3 

defining a transient error 13-1 

definition A-26 

error recording 12-14 

permanent error decision 13-5 

unsafe error decision 13-3 

used during phase one of error reporting sequence 12-7 

used with the Unsafe-Confinement-Area error type 12-10 
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Error-Record Register 

definition A-28 

error recording : 12-14 

initialization 14-3 

phase two of error reporting sequence 12-7 

Error- Reporting- Error Error Type 

definition 12-11 

determining which bus failed 13-18 

permanent error 12-15 

Error-Type Field 12-9 

F 

FAILURE Signal 3-32 

Fault-Tolerant I/O System 15-2 

FRC (Functional Redundancy Checking) 

description 11-3 

operation and setup 11-6 

overview 10-5 

physical connection 11-3 

FRC Register 

definition A-29 

FRC operation and setup 1 1-6 

used to marry processor modules using another module 13-8 

used to select a BXU 12-16 

FRC Splitting 13-26 

FRC-Splitting-Control Register 

definition A-30 

FRC operation and setup 1 1-6 

FRC splitting 13-26 

initialization 14-3 

used to distinguish between a warm or cold start 14-9 

FRF (Force Refresh) Signal 13-15 

FT1 Register 

BERL timing 12-2 

definition A-32 

example of use during a processor module marriage 14-29 

example of use during identification phase of initialization 14-24,14-27 

example of use during parameterization phase of initialization 14-19 

used by a memory controller interface 13-13 

used during IAC transactions .' 8-28,8-29,8-30 

used during initialization 14-5 

used during phase one of error reporting sequence 12-5 

used during the parameterization phase of initialization 14-8 

used for bus recovery 13-18 

used for processor module recovery 13-12 

used in an error type 12-12 

used with the attach-bus sequence 13-21 

FT2 Register 

definition A-34 

used during IAC transaction 8-29,8-30,8-32 

used during the parameterization phase of initialization 14-8 

used for bus recovery 13-19 
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used for processor module recovery 13-1 1 

used with the attach-bus sequence 13-21 

G 

Grant queue 7-24 

H 

HLDA (Hold Acknowledge) Signal) 3-17 

HLDAR (Hold Acknowledge Request) Signal 3-18 

HOLD Signal 3-17 

HOLDR (Hold Request) Signal 3-18 

I 

I/O Address Range 5-3 

I/O FRC Pair 15-2 

I/O Interface 

to a passive module 9-3 

to an active/passive module 9-4 

I/O interface to 8-bit and 16-bit Peripherals 5-1 

I/O Interface to the 80960MC 5-1 

I/O Prefetch Logic 8-34 

IAC Generator 

description 9-7 

design 9-9 

state diagram 9-11 

used for marrying a module shadow 14-11 

IAC Transactions 

address formats 7-17 

bus recovery ("Identify Device Order" IAC) 13-18 

bus recovery (Register Request Using a Logical Address" IAC) 13-18 

bus recovery ("Register Request Using a Physical Address" IAC) 13-19 

bus recovery (IAC Message) 13-18 

bus switching 13-16 

dataflow 7-17 

example of using the "Register Request From the L-Bus" IAC 14-18 

example of using the "Identify Device Order" IAC 14-20 

example of using the "Register Request From the L-Bus" IAC 14-31 

example of using the "Register Request Using a Logical Address" IAC 14-24,14-26 

example of using the "Register Request Using a Physical Address" IAC 14-22,14-28,14-25 

format for the "Identify Device Order" IAC 7-22 

format for the "Register Request Using a Physical Address" IAC 7-20 

format for the "Register Request Using a Logical Address" IAC 7-19 

format for the "Register Request From the L-Bus" IAC 7-21 

IAC address recognition (Logical) 8-28 

IAC address recognition (Message) 8-29,8-30 

IAC address recognition (Physical) 8-28 

IAC address recognition (Register) 8-29 

marrying processor modules employing "Register Request Using a Logical Address"IAC 13-10 

marrying processor modules employing "Register Request Using a Physical Address" IAC 13-7,13-9 

message 7-18 

module recovery using "Register Request Using a Physical Address IAC 13-1 1 

module recovery using the "Identify Device" IAC 13-1 1 
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operation after a bus switch 13-20 

retry sequence with IAC message 13-3 

ID-Parity Bit 12-10 

Identification Phase 

example , 14-20 

overview , 14-5 

Identify Device Order 

address format 7-22 

definition 7-22 

used during initialization 14-6 

See also IAC Transactions 

INIT-RAM 8-9, 14-5 

Initialization 

fbrtheM82965 14-2 

Initialization for 80960MC 3-30 

INITID (Initialization Identification) Signal 

connection between master and checker 14-31 

definition 7-5 

used during initialization 14-5 

INT /IAC (Interrupt or Inter- Agent Communication) 

Signal 3-7,3-28 

INT, (Interruptl) Signal 3-28 

INT 2 /INTR (Interrupt2 or Interrupt Request) 

Signal 3-25,3-26,3-28 

INT 3 /INTA (Interrupt3 or Interrupt Acknowledge) 

Signal 3-28 

Interleaving 8-28 

Interleaving Cache Memory 8-19 

Interrupts 

definition 3-24 

direct interrupt pins 3-26 

handling in fault-tolerant systems 15-6 

Interrupt Control register 3-26 

pins that interface to an interrupt controller 3-26 

signals 3-25 

synchronization 3-28 

timing diagram 3-27 

Invalid Bit 12-10 

Invalidate-Cache Command definition A-37 

used to synchronize caches 13-9 

L-Bus Interface Logic on the BXU functions, list of 8-9 

IAC address recognition (logical) 8-30 

IAC address recognition (Message) 8-30, 8-31 

IAC address recognition (Physical) 8-30 

IAC address recognition (Register) 8-29 

INIT-RAM address recognition 8-11 

L-bus and AP-bus conversions 8-1 1 

memory address recognition 8-11 

private memory address recognition 8-12 

L-Bus States address (T a ) 3-1, 3-14 

data(T d ) ! 3-1,3-16 

hold(T h ) 3-17 
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hold request (TJ 3-18 

idle(T.) .'. 3-1,3-16 

recovery (T) 3-1,3-16 

wait(T w ).J. 3-1,3-16 

LAD (Local Address/Data) lines 3-3 

LAD r LAD 

see SIZE signals 

LBI-Control Register 

arbitration usage 8-27 

cache configuration and control 8-18 

definition A-37 

example of use during parameterization phase of initialization 14-18 

example of use during a processor module marriage 14-28 

example of use during identification phase of initialization 14-24, 14-27 

interleaving after a bus switch 13-24 

state after RESET 14-5 

used for INIT-RAM address recognition 

ontheL-bus 8-10 

used for processor module recovery 13-12 

used to select Processor mode 14-3 

LERL,-LERL (Bus Error) signals 

used by the error reporting network 12-1 

used during phase one of error reporting sequence 12-5 

used during phase two of error reporting sequence 12-1 1 

Line 8-16 

Line-Valid 8-16 

Local-Bus-Test Register 

definition A-40 

diagnostic testing 8-39 

Lock Register 

definition A-42 

memory considerations after a bus switch 13-25 

used by a memory controller interface 13-13 

used by a memory module 13-12 

used during memory operation 8-27 

used during RMW operations 8-27 

LOCK Signal 

definition 3-6 

used during arbitration 3-18 

Logical-ID Register 

definition A-44 

example of use during a processor module marriage 14-22 

example of use during identification phase of initialization 14-20, 14-22, 14-23, 14-25, 14-26 

state after RESET 14-4 

used during IAC transactions 7-19, 7-20, 8-28, 8-29, 8-30 

used during the identification phase of initialization 14-7 

used to identify a BXU 7-2 

M 

Mask Register 

definition A-45 

example of use during parameterization phase of initialization 14-18 

example of use during a processor module marriage 14-28 
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used for bus recovery 13-19 

Match Register 

BXU operation when Sequence bit set to one 8-15 

definition A-45 

example of use during parameterization phase of initialization 14-18 

example of use during a processor module of marriage 14-29 

interleaving after a bus switch 13-24 

used for bus recovery 13-19 

used for bus switching 13-17 

used to marry processor modules using another module 13-10 

used with the attach-bus sequence 13-21 

Maxtime Register 

definition A-48 

transient wait period 10-7 

MEM/REG (Memory/Register) Signal 

definition 8-28 

used by a memory controller 13-14 

Memory Address Range 4-3 

Memory Controller and BXU 13-13 

Memory Interface to the BXU 9-1 

Memory Interface to the 80960MC 4-1 

Memory Mode 8-2, 8-27 

Memory Organization on AP-Bus 7-6 

Message Bus 8-29 

MODCHK (Module Check) Signal 

definition 7-5 

used during initialization 14-3 

usedinFRC 11-3 

Module Confinement Area 11-3 

Module Shadowing 

initialization example 14-17 

marriage using a special agent 14-11 

memory module definition 13-12, 13-14 

memory module marriage 13-15 

memory module response to error reports 13-12 

overview 13-7 

processor module definition 13-6 

processor module marriage 13-7, 13-15 

used for processor module recovery 13-11 

Module-Err or -ID Register 

definition A-49 

example of use during processor module marriage 14-29 

example of use during identification phase of initialization 14-22, 14-23, 14-25, 14-26 

example of use during parameterization phase of initialization 14-19 

FRC splitting 13-26 

permanent error decision 13-5 

used as the location in an error message 12-10, 12-11, 12-12, 12-13 

used by a memory controller interface 13-15 

used during initialization 14-5 

used for marrying modules by a special agent 14-12 

used in an error message 12-10, 12-11 

used to marry processor modules using another module 13-9 
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NACK (No-Acknowledgement) Reply 

definition 7-16 

used during IAC transactions 8-14 

used in IAC transactions 7-17, 7-20 

Null Bit 12-9 

P 

Packet 7-3 

Parameterization Phase 14-7 

Partial Write Operations 8-14, 13-14 

Partner Communication 10-7 

Passive Interface Circuit 15-4 

Passive Module 8-44 

PBM (Primary Bus Master) 3-20 

Permanent Error Decision ., 13-5 

Permanent Errors 13-3 

PFETCH (Prefetch) Signal 8-35 

Phase One of Error Report 12-5 

Phase Two of Error Report , 12-7 

Physical-ID Register 

definition A-50 

example of use during identification phase of initialization 14-20 

state after RESET 14-4 

used during IAC transactions 7-21, 8-30 

used during the identification phase of initialization , 14-5, 14-6 

POPQUE Signal 13-22 

Prefetch-Control Register 

considerations after a bus switch 13-26 

definition A-51 

example of use during a processor module marriage 14-29 

example of use during parameterization phase of initialization 14-19 

used for bus recovery 13-19 

used with the attach-bus sequence 13-21 

used with the I/O prefetch unit 8-34 

Primary-Catastrophe Error Type 

defining a permanent error 13-3 

defining a transient error 13-1 

definition 12-11 

unsafe error decision 13-3 

used during processor module recovery .. 13-1 1 

Primary-Catastrophe Command 

definition A-53 

when to use 12-11 

Primary-Elect Unit 13-9, 14-22 

Private Memory Mask Register 

definition A-54 

private memory address recognition 8-10 

Private Memory Match Register 

definition A-55 

private memory address recognition 8-12 

Processor Mode 8-2 
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Processor-Message Register 

used during IAC transactions 8-29 

used during message IAC requests 8-30, 8-31, 8-32 

Processor-Priority Register 

definition A-56 

used during IAC transactions 7-22, 8-27, 7-19 

used during message IAC requests 8-32, 8-33, 8-34 

write operations after a bus switch 13-20 

Q 

QMR (Quad Modular Redundancy) 

initialization example 14-18 

overview 10-5 

See also Module Shadowing 

QMR Register 

definition A-58 

example of use during a processor module marriage 14-28, 14-31 

example of use during identification phase of initialization 14-22, 14-23, 14-24, 14-26 

FRC splitting 13-27 

used during processor module recovery 13-11 

used for marrying modules by a special agent 14-11, 14-12 

used for processor module recovery 13-12 

used to marry processor modules using another module 13-10, 13-9 

used to select a BXU 12-16 

QMR I/O 15-6 

R 

Read Byte 

definition 7-13 

operation 8-13 

Read Double-Byte 

definition 7-13 

operation 8-13 

Read Operation 

retry operation 13-4 

sequence during prefetch operation 8-34 

sequence on the AP-bus 7-11 

Read Operation, timing diagram for the L-bus 3-9 

Read-Reply Packet 

operation 7-12 

used in RMW operations 7-15 

Read Request Packet 

general definition 7-11 

operation 7-11 

READY Signal 

definition 3-5 

timing diagram 3-7 

used by the 82586 interface 5-12 

used by the M8259A interface 5-7 

used by the 82786 interface 5-12, 5-13 

used by the BXU and cache 8-21 

used by the byte enable latch 4-5 

used by the SRAM interface 4-7, 4-9, 4-10 
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used by the timing control logic 4-5, 5-4 

with AP-bus transactions .; 8-13 

Recovery 13-1 

Redundancy 13-6 

Register Request from the L-Bus 
See IAC Transactions 

Register Request Using a Logical Address 
See IAC Transactions 

Register Request Using a Physical Address 
See IAC Transactions 

Register, a procedure for initialization 14-2 

Registers Listed in Appendix A 

Arbitration-ID A-12 

AP-Control A-8 

AP-Mask A-10 

AP-Match A-10 

Bus-Error-ID ...A-15 

Cache-Configuration A- 17 

Cache-Test A-21 

COM A-23 

Component-Specifier A-24 

Error-Log A-26 

Error-Record A-28 

FRC A-29 

FRC-Splitting-Control A-30 

FTI A-32 

FT2 A-34 

LBI-Control A-37 

Local-Bus-Test A-40 

Lock A-42 

Logical-ID A-44 

Mask • , A-46 

Maskj A-46 

Mask 2 A-46 

Match A-46 

Match j A-46 

Match 2 A-46 

Maxtime A-48 

Module-Error-ID A-49 

Physical-ID A-50 

Prefetch-Control A-51 

Private Memory Mask A-54 

Private Memory Match A-54 

Processor-Priority A-55 

Processor-Priority l A-55 

QMR A-57 

Spouse-ID A-62 

System-Bus-ID A-64 

Test-Detection .. A-66 

Test-Type A-69 

See also individual registers 

Reissue-reply packet 

definition 7-16 
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used in RMW operations 7-15 

Reply Ordering 7-31 

Reply Packet 

general definition 7-7 

operation on AP-bus 7-11 

types of 7-8 

Reporting and Recovery 10-7 

cycle 10-7 

Reporting Network 12-1 

Request Packet 

general definition 7-7 

operation on AP-bus 7-11 

types of 7-7 

RESET 

clock phase synchronization 14-2 

during default initialization phase 14-1 

FRC splitting 13-27 

signal on AP-bus 7-4 

timing generation for 80960MC 3-29 

used by the IAC generator 9-7 

Retry 10-8, 13-3 

RMW (Read-Modify-Write) Operation 

definition 7-15 

use for LOCK signal 8-13 

used during retry sequence 13-4 

RMW-Read Packet 7-15 

RMW- Write Packet 7-15 

RPYDEF (Reply Deferral) Signal 

definition 7-4,7-31 

used in FRC 11-5 

s 

SBM (Secondary Bus Master) 3-20 

Set 8-17 

Shadow-Catastrophe Error Type 

defining a permanent error 13-3 

defining a transient error 13-1 

definition 12-11 

unsafe error decision 13-3 

used during processor module recovery 13-1 1 

Shadow-Catastrophe Command 

definition A-61 

when to use 12-11 

Shadow-Elect Unit 13-9, 14-22 

SIZE Signals 

definition 3-3 

used by the 82586 interface 5-10 

used by the burst logic 4-3 

used by the prefetch unit of the BXU 8-36 

Slots 7-30 

Software Considerations for Reconfiguration 7-28 



-15 



iny 



INDEX 



Source-ID Field... 12-10 

SPEC f SPEC (Operation/Status) Signals 7-8 

SPEC 3 -SPEC 2 (Cycle Count) Signals 7-8 

SPEC 4 ( Multicycle) Signal 7-8 

SPEC 5 (REQUEST/REPLY) Signal 7-8 

SPEC 5 -SPEC (Specification) Signals 

definition 7-3 

used as byte marks 7-13 

used during read data transfers 7-1 

usedinFRC 11-4 

used with request and reply packets 7-8 

Spouse-ID Register 

definition A-62 

example of use during a processor module marriage 14-30 

example of use during identification phase of initialization 14-22, 14-23, 14-25, 14-26 

example of use during parameterization phase of initialization 14-19 

used for marrying modules by a special agent 14-12 

used for processor module recovery 13-11 

used to marry processor modules using another module 13-9 

SRAM Interface 

as cache with BXU 8-18,8-19 
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tHamilton/Avnet Electronics 
3510 Spring Forest Drive 
Raleigh 27604 
Tel: (919) 878-0819 
TWX: 510-928-1836 

Pioneer/Technologies Group, Inc. 
9801 A-Southern Pine Blvd. 
Charlotte 28210 
Tel: (919) 527-8188 
TWX: 810-621-0366 



OHIO 

Arrow Electronics, Inc. 
7620 McEwen Road 
Centerville 45459 
Tel: (513) 435-5563 
TWX: 810-459-1611 

tArrow Electronics, Inc. 
6236 Cochran Road 
Solon 44139 
Tel: (216) 248-3990 
TWX: 810-427-9409 

tHamilton/Avnet Electronics 
954 Senate Drive 
Dayton 45459 
Tel: (513) 439-6733 
TWX: 810-450-2531 

Hamilton/Avnet Electronics 
4588 Emery Industrial Pkwy. 
Warrensville Heights 44128 
Tel: (216) 349-5100 
TWX: 810-427-9452 

tHamilton/Avnet Electronics 
777 Brooksedge Blvd. 
Westerville 43081 
Tel: (614) 882-7004 

tPioneer Electronics 
4433 Interpoint Boulevard 
Dayton 45424 
Tel: (513) 236-9900 
TWX: 810-459-1622 

tPioneer Electronics 
4800 E. 131st Street 
Cleveland 44105 
Tel: (216) 587-3600 
TWX: 810-422-2211 



OKLAHOMA 

Arrow Electronics, Inc. 
1211 E. 51st St., Suite 101 
Tulsa 74146 
Tel: (918) 252-7537 

tHamilton/Avnet Electronics 
12121 E. 51st St., Suite 102A 
Tulsa 74146 
Tel: (918) 252-7297 



OREGON 

tAlmac Electronics Corp. 
1885 N.W. 169th Place 
Beaverton 97005 
Tel: (503) 629-8090 
TWX: 910-467-8746 

tHamilton/Avnet Electronics 
6024 S.W. Jean Road 
Bldg. C, Suite 10 
Lake Oswego 97034 
Tel: (503) 635-7848 
TWX: 910-455-8179 

Wyle Distribution Group 

5250 N.E. Elam Young Parkway 

Suite 600 

Hillsboro 97124 

Tel: (503) 640-6000 

TWX: 910-460-2203 



PENNSYLVANIA 

Arrow Electronics, Inc. 
650 Seco Road 
Monroeville 15146 
Tel: (412) 856-7000 

Hamilton/Avnet Electronics 
2800 Liberty Ave. 
Pittsburgh 15238 
Tel: (412) 281-4150 

Pioneer Electronics 
259 Kappa Drive 
Pittsburgh 15238 
Tel: (412) 782-2300 
TWX: 710-795-3122 

tPioneer/Technologies Group, Inc. 

Delaware Valley 

261 Gibralter Road 

Horsham 19044 

Tel: (215)674-4000 

TWX: 510-665-6778 



TEXAS 

tArrow Electronics, Inc. 
3220 Commander Drive 
Carrollton 75006 
Tel: (214) 380-6464 
TWX: 910-860-5377 

tArrow Electronics, Inc. 
10899 Kinghurst 
Suite 100 
Houston 77099 
Tel: (713) 530-4700 
TWX: 910-880-4439 

tArrow Electronics, Inc. 
2227 W. Braker Lane 
Austin 78758 
Tel: (512) 835-4180 
TWX: 910-874-1348 

tHamilton/Avnet Electronics 
1807 W. Braker Lane 
Austin 78758 
Tel: (512)837-8911 
TWX: 910-874-1319 

tHamilton/Avnet Electronics 
2111 W. Walnut Hill Lane 
Irving 75038 
Tel: (214) 550-6111 
TWX: 910-860-5929 



tHamilton/Avnet Electronics 
4850 Wright Rd., Suite 190 
Stafford 77477 
Tel: (713) 240-7733 
TWX: 910-881-5523 

tPioneer Electronics 
18260 Kramer 
Austin 78758 
Tel: (512) 835-4000 
TWX: 910-874-1323 

tPioneer Electronics 
13710 Omega Road 
Dallas 75234 
Tel: (214)386-7300 
TWX: 910-850-5563 

tPioneer Electronics 
5853 Point West Drive 
Houston 77036 
Tel: (713) 988-5555 
TWX: 910-881-1606 

Wyle Distribution Group 
1810 Greenville Avenue 
Richardson 75081 
Tel: (214)235-9953 



UTAH 

Arrow Electronics 
1946 Parkway Blvd. 
Salt Lake City 841 19 
Tel: (801) 973-6913 

tHamilton/Avnet Electronics 
1585 West 2100 South 
Salt Lake City 84119 
Tel: (801) 972-2800 
TWX: 910-925-4018 

Wyle Distribution Group 
1325 West 2200 South 
Suite E 

West Valley 841 19 
Tel: (801) 974-9953 



WASHINGTON 

tAlmac Electronics Corp. 
14360 S.E. Eastgate Way 
Bellevue 98007 
Tel: (206) 643-9992 
TWX: 910-444-2067 

Arrow Electronics, Inc. 
19540 68th Ave. South 
Kent 98032 
Tel: (206) 575-4420 

tHamilton/Avnet Electronics 
14212 N.E. 21st Street 
Bellevue 98005 
Tel: (206) 643-3950 
TWX: 910-443-2469 

Wyle Distribution Group 
15385 N.E. 90th Street 
Redmond 98052 
Tel: (206)881-1150 



WISCONSIN 

Arrow Electronics, Inc. 

200 N. Patrick Blvd., Ste. 100 

Brookfield 53005 

Tel: (414) 767-6600 

TWX: 910-262-1193 

Hamilton/Avnet Electronics 
2975 Moorland Road 
New Berlin 53151 
Tel: (414) 784-4510 
TWX: 910-262-1182 



CANADA 

ALBERTA 

Hamilton/Avnet Electronics 

2816 21st Street N.E. 

Calgary T2E 6Z3 

Tel: (403) 230-3586 

TWX: 03-827-642 

Zentronics 

Bay No. 1 

3300 14th Avenue N.E. 

Calgary T2A 6J4 

Tel: (403) 272-1021 

BRITISH COLUMBIA 

tHamilton/Avnet Electronics 

105-2550 Boundary 

Burmalay V5M 3Z3 

Tel: (604) 437-6667 

Zentronics 

108-11400 Bridgeport Road 

Richmond V6X 1T2 

Tel: (604) 273-5575 

TWX: 04-5077-89 

MANITOBA 

Zentronics 

60-1313 Border Unit 60 
Winnipeg R3H 0X4 
Tel: (204) 694-1957 

ONTARIO 

Arrow Electronics, Inc. 
36 Antares Dr. 
Nepean K2E 7W5 
Tel: (613) 226-6903 
Arrow Electronics, Inc. 
1093 Meyerside 
Mississauga L5T 1M4 
Tel: (416) 673-7769 
TWX: 06-218213 
tHamilton/Avnet Electronics 
6845 Rexwood Road 
Units 3-4-5 
Mississauga L4T 1 R2 
Tel: (416) 677-7432 
TWX: 610-492-8867 
Hamilton/Avnet Electronics 
6845 Rexwood Rd., Unit 6 
Mississauga L4T 1 R2 
Tel: (416) 277-0484 
tHamilton/Avnet Electronics 
190 Colonnade Road South 
Nepean K2E 7L5 
Tel: (613) 226-1700 
TWX: 05-349-71 

tZentronics 
8 Tilbury Court 
Brampton L6T 3T4 
Tel: (416) 451-9600 
TWX: 06-976-78 
tZentronics 
1 55 Colonnade Road 
Unit 17 

Nepean K2E 7K1 
Tel: (613) 226-8840 
Zentronics 
60-1313 Border St. 
Winnipeg R3H 0I4 
Tel: (204) 694-7957 

QUEBEC 

tArrow Electronics Inc. 

4050 Jean Talon Quest 

Montreal H4P 1W1 

Tel: (514) 735-5511 

TWX: 05-25590 

Arrow Electronics, Inc. 

500 Avenue St-Jean Baptiste 

Suite 280 

Quebec G2E 5R9 

Tel: (418)871-7500 

FAX: 418-871-6816 

Hamilton/Avnet Electronics 

2795 Halpern 

St. Laurent H2E 7K1 

Tel: (514)335-1000 

TWX: 610-421-3731 

Zentronics 
817 McCaffrey 
St. Laurent H4T 1 M3 
Tel: (514) 737-9700 
TWX: 05-827-535 



tMicrocomputer System Technical Distributor Center 



Intel 



EUROPEAN SALES OFFICES 



DENMARK 

Intel Denmark A/S 
Glentevej 61, 3rd Floor 
2400 Copenhagen NV 
Tel: (45) (31) 19 80 33 
TLX: 19567 

FINLAND 

Intel Finland OY 
Ruosilantie 2 
00390 Helsinki 
Tel: (358) 544 644 
TLX: 123332 

FRANCE 

Intel Corporation S.A.R.L. 

1 , Rue Edison-BP 303 

78054 St. Quentin-en-Yvelines 

Cedex 

Tel: (33) (1) 30 57 70 00 

TLX: 699016 



WEST GERMANY 

Intel Semiconductor GmbH* 

Dornacher Strasse 1 

8016 Feldkirchen bei Muenchen 

Tel: (49) 089/90992-0 

TLX: 5-23177 

FAX: 904-3948 

Intel Semiconductor GmbH 
Hohenzollern Strasse 5 
3000 Hannover 1 
Tel: (49)0511/344081 
TLX: 9-23625 

Intel Semiconductor GmbH 
Abraham Lincoln Strasse 16-18 
6200 Wiesbaden 
Tel: (49) 06121/7605-0 
TLX: 4-186183 



Intel Semiconductor GmbH 

Zettachring 10A 

7000 Stuttgart 80 

Tel: (49)0711/7287-280 

TLX: 7-254826 

ISRAEL 

Intel Semiconductor Ltd.* 

Atidim Industrial Park-Neve Sharet 

P.O. Box 43202 

Tel-Aviv 61430 

Tel: (972) 03-498080 

TLX: 371215 



ITALY 

Intel Corporation Italia S.p.A.* 

Milanofiori Palazzo E 

20090 Assago 

Milano 

Tel: (39) (02) 89200950 

TLX: 341286 



NETHERLANDS 

Intel Semiconductor B.V.* 
Postbus 84130 
3099 CC Rotterdam 
Tel: (31) 10.407.11.11 
TLX: 22283 

NORWAY 

Intel Norway A/S 

Hvamveien 4-PO Box 92 

2013Skjetten 

Tel: (47) (6) 842 420 

TLX: 78018 

SPAIN 

Intel Iberia S.A. 
Zurbaran, 28 
28010 Madrid 
Tel: (34) 308.25.52 
TLX: 46880 



SWEDEN 

Intel Sweden A. B.* 

Dalvagen 24 

171 36Solna 

Tel: (46) 8 734 01 00 

TLX: 12261 



SWITZERLAND 

Intel Semiconductor A.G. 

Zuerichstrasse 

8185 Winkel-Rueti bei Zuerich 

Tel: (41) 01/860 62 62 

TLX: 825977 

UNITED KINGDOM 

Intel Corporation (U.K.) Ltd.* 
Pipers Way 

Swindon, Wiltshire SN3 1RJ 
Tel: (44) (0793) 696000 
TLX: 444447/8 



EUROPEAN DISTRIBUTORS/REPRESENTATIVES 



AUSTRIA 

Bacher Electronics G.m.b.H. 

Rotenmuehlgasse 26 

1120Wien 

Tel: (43) (0222) 83 56 46 

TLX: 31532 

BELGIUM 

Inelco Belgium S.A. 

Av. des Croix de Guerre 94 

1120 Bruxelles 

Oorlogskruisenlaan, 94 

1120 Brussel 

Tel: (32) (02) 216 01 60 

TLX: 64475 or 22090 

DENMARK 

ITT- M u Iti kom ponent 

Naverland 29 

2600 Glostrup 

Tel: (45) (0) 2 45 66 45 

TLX: 33 355 

FINLAND 

OY Fintronic AB 
Melkonkatu 24A 
00210 Helsinki 
Tel: (358) (0) 6926022 
TLX: 124224 

FRANCE 

Almex 

Zone industrielle d'Antony 

48, rue de I'Aubepine 

BP 102 

92164 Antony cedex 

Tel: (33) (1)46 66 21 12 

TLX: 250067 

Jermyn-Generim 
60, rue des Gemeaux 
Silic 580 

94653 Rungis cedex 
Tel: (33) (1)49 78 49 78 
TLX: 260967 

Metrologie 
Tour d'Asnieres 
4, av. Laurent-Cely 
92606 Asnieres Cedex 
Tel: (33) (1)47 90 62 40 
TLX: 611448 



Tekelec-Airtronic 
Cite des Bruyeres 
Rue Carle Vernet - BP 2 
92310 Sevres 
Tel: (33) (1)45 34 75 35 
TLX: 204552 

WEST GERMANY 

Electronic 2000 AG 
Stahlgruberring 12 
8000 Muenchen 82 
Tel: (49) 089/42001-0 
TLX: 522561 

ITT Multikomponent GmbH 
Postfach 1265 
Bahnhofstrasse 44 
7141 Moeglingen 
Tel: (49)07141/4879 
TLX: 7264472 

Jermyn GmbH 
Im Dachsstueck 9 
6250 Limburg 
Tel: (49) 06431/508-0 
TLX: 415257-0 

Metrologie GmbH 
Meglingerstrasse 49 
8000 Muenchen 71 
Tel: (49) 089/78042-0 
TLX: 5213189 

Proelectron Vertriebs GmbH 
Max Planck Strasse 1 -3 
6072 Dreieich 
Tel: (49) 06103/30434-3 
TLX: 417903 

IRELAND 

Micro Marketing Ltd. 

Glenageary Office Park 

Glenageary 

Co. Dublin 

Tel: (21) (353) (01)85 63 25 

TLX: 31584 

ISRAEL 

Eastronics Ltd. 
1 1 Rozanis Street 
P.O.B. 39300 
Tel-Aviv 61392 
Tel: (972)03-475151 
TLX: 33638 



ITALY 

Intesi 

Divisione ITT Industries GmbH 

Viale Milanofiori 

Palazzo E/5 

20090 Assago (Ml) 

Tel: (39) 02/824701 

TLX: 311351 

Lasi Elettronica S.p.A. 
V. le FulvioTesti, 126 
20092 Cinisello Balsamo (Ml) 
Tel: (39)02/2440012 
TLX: 352040 

ITT Multicomponents 
Viale Milanofiori E/5 
20090 Assago (Ml) 
Tel: (39) 02/824701 
TLX: 311351 

Silverstar 

Via Dei Gracchi 20 
20146 Milano 
Tel: (39) 02/49961 
TLX: 332189 

NETHERLANDS 

Koning en Hartman Elektrotechniek 

B.V. 

Energieweg 1 

2627 AP Delft 

Tel: (31) (0) 15/609906 

TLX: 38250 

NORWAY 

Nordisk Elektronikk (Norge) A/S 

Postboks 123 

Smedsvingen 4 

1364Hvalstad 

Tel: (47) (02) 84 62 10 

TLX: 77546 

PORTUGAL 

ATD Portugal 

Rua Dos Lusiadas, 5 Sala B 

1300 LISBOA 

Tel: 64 80 91 

TLX: 61562 

FAX: 63 76 55 



Ditram 

Avenida Miguel Bombarda, 133 

1000 Lisboa 

Tel: (35) (1) 734 884 

TLX: 14182 

SPAIN 

ATD Electronica, S.A. 
Plaza Ciudad de Viena, 6 
28040 Madrid 
Tel: (34) (1) 234 40 00 
TLX: 42754 

ITT-SESA 

Calle Miguel Angel, 21-3 

28010 Madrid 

Tel: (34) (1)419 54 00 

TLX: 27461 

Metrologia Iberica, S.A. 
Ctra. de Fuencarral, n.80 
28100 Alcobendas (Madrid) 
Tel: (34) (1) 653 86 11 

SWEDEN 

Nordisk Elektronik AB 

Huvudstagatan 1 

Box 1409 

171 27Solna 

Tel: (46) 08-734 97 70 

TLX: 105 47 

SWITZERLAND 

Industrade A.G. 
Hertistrasse 31 
8304 Wallisellen 
Tel: (41) (01) 8328111 
TLX: 56788 

TURKEY 

EMPA Electronic 
Lindwurmstrasse 95A 
8000 Muenchen 2 
Tel: (49) 089/53 80 570 
TLX: 528573 

UNITED KINGDOM 

Accent Electronic Components Ltd. 
Jubilee House, Jubilee Road 
Letchworth, Herts SG6 1TL 
Tel: (44) (0462) 686666 
TLX: 826293 



Bytech-Comway Systems 
3 The Western Centre 
Western Road 
Bracknell RG12 1RW 
Tel: (44) (0344) 55333 
TLX: 847201 

Jermyn 

Vestry Estate 

Otford Road 

Sevenoaks 

KentTN14 5EU 

Tel: (44) (0732) 450144 

TLX: 95142 

MMD 

Unit 8 Southview Park 

Caversham 

Reading 

Berkshire RG4 OAF 

Tel: (44) (0734) 481666 

TLX: 846669 

Rapid Silicon 
Rapid House 
Denmark Street 
High Wycombe 
Buckinghamshire HP1 1 2ER 
Tel: (44) (0494) 442266 
TLX: 837931 

Rapid Systems 
Rapid House 
Denmark Street 
High Wycombe 
Buckinghamshire HP11 2ER 
Tel: (44) (0494) 450244 
TLX: 837931 



YUGOSLAVIA 

H.R. Microelectronics Corp. 

2005 de la Cruz Blvd., Ste. 22C 

Santa Clara, CA 95050 

U.S.A. 

Tel: (1) (408) 988-0286 

TLX: 387452 

Rapido Electronic Components 

S.p.a. 

Via C. Beccaria, 8 

34133 Trieste 

Italia 

Tel: (39) 040/360555 

TLX: 460461 



♦Field Application Location 



UNITED STATES 
Intel Corporation 
3065 Bowers Avenue 
Santa Clara, CA 95051 



JAPAN 

Intel Japan K.K. 

5-6 Tokodai, Tsukuba-shi 

Ibaraki, 300-26 






FRANCE 

Intel Corporation S . A . R . L . 

1, Rue Edison, BP 303 

78054 Saint-Quentin-en-Yvelines Cedex 

UNITED KINGDOM 

Intel Corporation (U.K.) Ltd . 

Pipers Way 

Swindon 

Wiltshire, England SN3 1RJ 

WEST GERMANY 

Intel Semiconductor GmbH 

Dornacher Strasse 1 

8016 Feldkirchen bei Muenchen 






HONG KONG 
Intel Semiconductor Ltd 
10/F East Tower 
Bond Center 
Queensway, Central 






CANADA 

Intel Semiconductor of Canada, Ltd 
190 Attwell Drive, Suite 500 
Rexdale, Ontario M9W 6H8 
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